Este documento está basado en el manterial preparado por INPE para su curso de satélites geoestacionarios del GOES-16. Se puede encontrar el video de youtube aquí.
!cat /etc/issue
!uname -a
Ubuntu 18.04.5 LTS \n \l Linux 098ff6e77b56 4.19.112+ #1 SMP Thu Jul 23 08:00:38 PDT 2020 x86_64 x86_64 x86_64 GNU/Linux
!python --version
Python 3.7.10
pip install --upgrade pip
Collecting pip
Downloading https://files.pythonhosted.org/packages/fe/ef/60d7ba03b5c442309ef42e7d69959f73aacccd0d86008362a681c4698e83/pip-21.0.1-py3-none-any.whl (1.5MB)
|████████████████████████████████| 1.5MB 5.0MB/s
Installing collected packages: pip
Found existing installation: pip 19.3.1
Uninstalling pip-19.3.1:
Successfully uninstalled pip-19.3.1
Successfully installed pip-21.0.1
Se necesita tener netCDF4 para poder abrir los archivos que contienen datos satelitales.
pip install netcdf4
Collecting netcdf4
Downloading netCDF4-1.5.6-cp37-cp37m-manylinux2014_x86_64.whl (4.7 MB)
|████████████████████████████████| 4.7 MB 5.2 MB/s
Requirement already satisfied: numpy>=1.9 in /usr/local/lib/python3.7/dist-packages (from netcdf4) (1.19.5)
Collecting cftime
Downloading cftime-1.4.1-cp37-cp37m-manylinux2014_x86_64.whl (313 kB)
|████████████████████████████████| 313 kB 35.3 MB/s
Installing collected packages: cftime, netcdf4
Successfully installed cftime-1.4.1 netcdf4-1.5.6
Se necesita tener Cartopy para poder generar mapas en de las visualizaciones.
pip uninstall -y shapely imgaug
Found existing installation: Shapely 1.7.1 Uninstalling Shapely-1.7.1: Successfully uninstalled Shapely-1.7.1 Found existing installation: imgaug 0.2.9 Uninstalling imgaug-0.2.9: Successfully uninstalled imgaug-0.2.9
pip install cartopy
Collecting cartopy
Downloading Cartopy-0.18.0.tar.gz (14.4 MB)
|████████████████████████████████| 14.4 MB 175 kB/s
Requirement already satisfied: numpy>=1.10 in /usr/local/lib/python3.7/dist-packages (from cartopy) (1.19.5)
Collecting shapely>=1.5.6
Downloading Shapely-1.7.1-cp37-cp37m-manylinux1_x86_64.whl (1.0 MB)
|████████████████████████████████| 1.0 MB 50.9 MB/s
Collecting pyshp>=1.1.4
Downloading pyshp-2.1.3.tar.gz (219 kB)
|████████████████████████████████| 219 kB 60.2 MB/s
Requirement already satisfied: six>=1.3.0 in /usr/local/lib/python3.7/dist-packages (from cartopy) (1.15.0)
Requirement already satisfied: setuptools>=0.7.2 in /usr/local/lib/python3.7/dist-packages (from cartopy) (54.2.0)
Building wheels for collected packages: cartopy, pyshp
Building wheel for cartopy (setup.py) ... done
Created wheel for cartopy: filename=Cartopy-0.18.0-cp37-cp37m-linux_x86_64.whl size=15127818 sha256=1499e36e300db62a6d3bef668e71c951daef4b76dc6b12cb48457ed74a77735d
Stored in directory: /root/.cache/pip/wheels/0b/a9/54/172056df34478378e0636d30cd4d1a868de00e37254649bf1a
Building wheel for pyshp (setup.py) ... done
Created wheel for pyshp: filename=pyshp-2.1.3-py3-none-any.whl size=37263 sha256=d7a6dd05e74f3bec13bb5508f265ce273993581bc0cfa9dc8849b30adcc39f71
Stored in directory: /root/.cache/pip/wheels/43/f8/87/53c8cd41545ba20e536ea29a8fcb5431b5f477ca50d5dffbbe
Successfully built cartopy pyshp
Installing collected packages: shapely, pyshp, cartopy
Successfully installed cartopy-0.18.0 pyshp-2.1.3 shapely-1.7.1
pip install shapely --no-binary shapely --force
Collecting shapely
Downloading Shapely-1.7.1.tar.gz (383 kB)
|████████████████████████████████| 383 kB 3.8 MB/s
Skipping wheel build for shapely, due to binaries being disabled for it.
Installing collected packages: shapely
Attempting uninstall: shapely
Found existing installation: Shapely 1.7.1
Uninstalling Shapely-1.7.1:
Successfully uninstalled Shapely-1.7.1
Running setup.py install for shapely ... done
Successfully installed shapely-1.7.1
Se necesita tener Boto3 para...
pip install boto3
Collecting boto3
Downloading boto3-1.17.51-py2.py3-none-any.whl (131 kB)
|████████████████████████████████| 131 kB 4.4 MB/s
Collecting botocore<1.21.0,>=1.20.51
Downloading botocore-1.20.51-py2.py3-none-any.whl (7.4 MB)
|████████████████████████████████| 7.4 MB 5.6 MB/s
Collecting jmespath<1.0.0,>=0.7.1
Downloading jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.4.0,>=0.3.0
Downloading s3transfer-0.3.7-py2.py3-none-any.whl (73 kB)
|████████████████████████████████| 73 kB 1.1 MB/s
Collecting urllib3<1.27,>=1.25.4
Downloading urllib3-1.26.4-py2.py3-none-any.whl (153 kB)
|████████████████████████████████| 153 kB 59.1 MB/s
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/local/lib/python3.7/dist-packages (from botocore<1.21.0,>=1.20.51->boto3) (2.8.1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.21.0,>=1.20.51->boto3) (1.15.0)
Installing collected packages: urllib3, jmespath, botocore, s3transfer, boto3
Attempting uninstall: urllib3
Found existing installation: urllib3 1.24.3
Uninstalling urllib3-1.24.3:
Successfully uninstalled urllib3-1.24.3
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
requests 2.23.0 requires urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1, but you have urllib3 1.26.4 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
Successfully installed boto3-1.17.51 botocore-1.20.51 jmespath-0.10.0 s3transfer-0.3.7 urllib3-1.26.4
Se necesita actualizar gdal para...
!dpkg -l | grep libgdal
ii libgdal-dev 2.2.3+dfsg-2 amd64 Geospatial Data Abstraction Library - Development files ii libgdal20 2.2.3+dfsg-2 amd64 Geospatial Data Abstraction Library
!ogrinfo --version
GDAL 2.2.3, released 2017/11/20
!apt-add-repository -y ppa:ubuntugis/ubuntugis-unstable
!add-apt-repository -y ppa:ubuntugis/ppa
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] Get:2 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran40/ InRelease [3,626 B] Ign:3 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease Get:4 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic InRelease [15.9 kB] Hit:5 http://archive.ubuntu.com/ubuntu bionic InRelease Ign:6 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 InRelease Get:7 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 Release [697 B] Hit:8 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 Release Get:9 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 Release.gpg [801 B] Get:10 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] Hit:11 http://ppa.launchpad.net/cran/libgit2/ubuntu bionic InRelease Get:12 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [372 kB] Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,404 kB] Get:14 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] Get:15 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic InRelease [15.9 kB] Get:16 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [2,077 kB] Hit:17 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease Get:18 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic InRelease [20.8 kB] Ign:20 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 Packages Get:20 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 Packages [634 kB] Get:21 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main Sources [1,751 kB] Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [2,175 kB] Get:23 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 Packages [896 kB] Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [402 kB] Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [2,508 kB] Get:26 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic/main amd64 Packages [39.5 kB] Get:27 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 Packages [67.1 kB] Fetched 12.6 MB in 4s (2,855 kB/s) Reading package lists... Done Ign:1 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease Hit:2 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran40/ InRelease Hit:3 http://archive.ubuntu.com/ubuntu bionic InRelease Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease Hit:5 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic InRelease Ign:6 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 InRelease Hit:7 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 Release Hit:8 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 Release Hit:9 http://archive.ubuntu.com/ubuntu bionic-updates InRelease Hit:10 http://archive.ubuntu.com/ubuntu bionic-backports InRelease Hit:11 http://ppa.launchpad.net/cran/libgit2/ubuntu bionic InRelease Hit:12 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic InRelease Hit:13 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease Get:14 http://ppa.launchpad.net/ubuntugis/ppa/ubuntu bionic InRelease [20.7 kB] Hit:15 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic InRelease Get:18 http://ppa.launchpad.net/ubuntugis/ppa/ubuntu bionic/main amd64 Packages [69.1 kB] Fetched 89.8 kB in 4s (24.5 kB/s) Reading package lists... Done
!apt update
Ign:1 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease Hit:2 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran40/ InRelease Ign:3 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 InRelease Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease Hit:5 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic InRelease Hit:6 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 Release Hit:7 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 Release Hit:8 http://archive.ubuntu.com/ubuntu bionic InRelease Hit:9 http://archive.ubuntu.com/ubuntu bionic-updates InRelease Hit:10 http://ppa.launchpad.net/cran/libgit2/ubuntu bionic InRelease Hit:11 http://archive.ubuntu.com/ubuntu bionic-backports InRelease Hit:12 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic InRelease Hit:13 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease Hit:14 http://ppa.launchpad.net/ubuntugis/ppa/ubuntu bionic InRelease Hit:15 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic InRelease Reading package lists... Done Building dependency tree Reading state information... Done 94 packages can be upgraded. Run 'apt list --upgradable' to see them.
!apt upgrade
Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following packages were automatically installed and are no longer required: libgeos-3.6.2 linux-headers-4.15.0-140 linux-headers-4.15.0-140-generic r-cran-covr r-cran-crosstalk r-cran-dt r-cran-htmlwidgets r-cran-later r-cran-lazyeval r-cran-promises r-cran-rex Use 'apt autoremove' to remove them. The following NEW packages will be installed: libgeos-3.8.0 libgeotiff5 libproj13 libproj19 linux-headers-4.15.0-141 linux-headers-4.15.0-141-generic proj-bin The following packages have been kept back: gdal-bin gdal-data libcudnn8 libcudnn8-dev libgdal-dev libgdal20 libnccl-dev libnccl2 libopencv-calib3d-dev libopencv-calib3d3.2 libopencv-contrib-dev libopencv-contrib3.2 libopencv-core-dev libopencv-core3.2 libopencv-dev libopencv-features2d-dev libopencv-features2d3.2 libopencv-flann-dev libopencv-flann3.2 libopencv-highgui-dev libopencv-highgui3.2 libopencv-imgcodecs-dev libopencv-imgcodecs3.2 libopencv-imgproc-dev libopencv-imgproc3.2 libopencv-ml-dev libopencv-ml3.2 libopencv-objdetect-dev libopencv-objdetect3.2 libopencv-photo-dev libopencv-photo3.2 libopencv-shape-dev libopencv-shape3.2 libopencv-stitching-dev libopencv-stitching3.2 libopencv-superres-dev libopencv-superres3.2 libopencv-ts-dev libopencv-video-dev libopencv-video3.2 libopencv-videoio-dev libopencv-videoio3.2 libopencv-videostab-dev libopencv-videostab3.2 libopencv-viz-dev libopencv-viz3.2 libopencv3.2-java libopencv3.2-jni libvtk6.3 python-gdal The following packages will be upgraded: binutils binutils-common binutils-x86-64-linux-gnu cuda-compat-11-0 gnupg2 libaudit-common libaudit1 libbinutils libc-bin libcublas-dev libcublas10 libcudnn7 libcudnn7-dev libgeos-c1v5 libgeos-dev libgeotiff-dev libgeotiff2 libhogweed4 libldap-2.4-2 libldap-common libnettle6 libp11-kit0 libperl5.26 libproj-dev libsasl2-2 libsasl2-modules-db libseccomp2 libspatialite-dev libspatialite7 libzstd1 linux-headers-generic linux-libc-dev openssl perl perl-base perl-modules-5.26 proj-data r-cran-dbplyr r-cran-devtools r-cran-pkgload tar ubuntu-keyring xserver-common xserver-xorg-core-hwe-18.04 44 upgraded, 7 newly installed, 0 to remove and 50 not upgraded. Need to get 499 MB of archives. After this operation, 276 MB of additional disk space will be used. Get:1 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 cuda-compat-11-0 450.102.04-1 [6,712 kB] Get:2 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libgeos-3.8.0 amd64 3.8.0-1~bionic0 [541 kB] Get:3 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 libcublas10 10.2.3.254-1 [43.1 MB] Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libperl5.26 amd64 5.26.1-6ubuntu0.5 [3,534 kB] Get:5 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libgeos-dev amd64 3.8.0-1~bionic0 [96.8 kB] Get:6 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 libcublas-dev 10.2.3.254-1 [42.4 MB] Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 perl amd64 5.26.1-6ubuntu0.5 [201 kB] Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 perl-base amd64 5.26.1-6ubuntu0.5 [1,391 kB] Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 perl-modules-5.26 all 5.26.1-6ubuntu0.5 [2,762 kB] Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 tar amd64 1.29b-2ubuntu0.2 [234 kB] Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libc-bin amd64 2.27-3ubuntu1.4 [643 kB] Get:12 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libgeos-c1v5 amd64 3.8.0-1~bionic0 [76.6 kB] Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaudit-common all 1:2.8.2-1ubuntu1.1 [4,068 B] Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaudit1 amd64 1:2.8.2-1ubuntu1.1 [38.7 kB] Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libzstd1 amd64 1.3.3+dfsg-2ubuntu1.2 [189 kB] Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnettle6 amd64 3.4-1ubuntu0.1 [110 kB] Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libhogweed4 amd64 3.4-1ubuntu0.1 [137 kB] Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libp11-kit0 amd64 0.23.9-2ubuntu0.1 [187 kB] Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libseccomp2 amd64 2.5.1-1ubuntu1~18.04.1 [43.1 kB] Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ubuntu-keyring all 2018.09.18.1~18.04.2 [22.3 kB] Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 openssl amd64 1.1.1-1ubuntu2.1~18.04.9 [614 kB] Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 binutils-x86-64-linux-gnu amd64 2.30-21ubuntu1~18.04.5 [1,839 kB] Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 binutils-common amd64 2.30-21ubuntu1~18.04.5 [197 kB] Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 binutils amd64 2.30-21ubuntu1~18.04.5 [3,388 B] Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libbinutils amd64 2.30-21ubuntu1~18.04.5 [489 kB] Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsasl2-modules-db amd64 2.1.27~101-g0780600+dfsg-3ubuntu2.3 [15.0 kB] Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsasl2-2 amd64 2.1.27~101-g0780600+dfsg-3ubuntu2.3 [49.2 kB] Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libldap-common all 2.4.45+dfsg-1ubuntu1.10 [15.8 kB] Get:29 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libldap-2.4-2 amd64 2.4.45+dfsg-1ubuntu1.10 [154 kB] Get:30 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-141 all 4.15.0-141.145 [10.9 MB] Get:31 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 proj-data all 7.0.0-1~bionic0 [7,707 kB] Get:32 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-141-generic amd64 4.15.0-141.145 [1,266 kB] Get:33 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.141.128 [2,484 B] Get:34 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-libc-dev amd64 4.15.0-141.145 [998 kB] Get:35 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 xserver-common all 2:1.19.6-1ubuntu4.9 [26.8 kB] Get:36 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 xserver-xorg-core-hwe-18.04 amd64 2:1.20.8-2ubuntu2.2~18.04.5 [1,334 kB] Get:37 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 gnupg2 all 2.2.4-1ubuntu1.4 [5,292 B] Get:38 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 libcudnn7-dev 7.6.5.32-1+cuda10.2 [165 MB] Get:39 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libproj19 amd64 7.0.0-1~bionic0 [925 kB] Get:40 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libgeotiff5 amd64 1.5.1-2~bionic2 [73.6 kB] Get:41 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libproj-dev amd64 7.0.0-1~bionic0 [10.8 MB] Get:42 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libgeotiff-dev amd64 1.5.1-2~bionic2 [100 kB] Get:43 http://ppa.launchpad.net/ubuntugis/ppa/ubuntu bionic/main amd64 libproj13 amd64 5.2.0-1~bionic0 [202 kB] Get:44 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 libcudnn7 7.6.5.32-1+cuda10.2 [189 MB] Get:45 http://ppa.launchpad.net/ubuntugis/ppa/ubuntu bionic/main amd64 libgeotiff2 amd64 1.4.2-2build1+bionic0 [73.7 kB] Get:46 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libspatialite-dev amd64 4.3.0a-6~bionic3 [1,364 kB] Get:47 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libspatialite7 amd64 4.3.0a-6~bionic3 [1,240 kB] Get:48 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 proj-bin amd64 7.0.0-1~bionic0 [158 kB] Get:49 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 r-cran-dbplyr all 2.1.1-1cran1.1804.0 [768 kB] Get:50 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 r-cran-pkgload all 1.2.1-1cran1.1804.0 [151 kB] Get:51 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 r-cran-devtools all 2.4.0-1cran1.1804.0 [377 kB] Fetched 499 MB in 10s (49.5 MB/s) Extracting templates from packages: 100% (Reading database ... 160983 files and directories currently installed.) Preparing to unpack .../libperl5.26_5.26.1-6ubuntu0.5_amd64.deb ... Unpacking libperl5.26:amd64 (5.26.1-6ubuntu0.5) over (5.26.1-6ubuntu0.3) ... Preparing to unpack .../perl_5.26.1-6ubuntu0.5_amd64.deb ... Unpacking perl (5.26.1-6ubuntu0.5) over (5.26.1-6ubuntu0.3) ... Preparing to unpack .../perl-base_5.26.1-6ubuntu0.5_amd64.deb ... Unpacking perl-base (5.26.1-6ubuntu0.5) over (5.26.1-6ubuntu0.3) ... Setting up perl-base (5.26.1-6ubuntu0.5) ... (Reading database ... 160983 files and directories currently installed.) Preparing to unpack .../perl-modules-5.26_5.26.1-6ubuntu0.5_all.deb ... Unpacking perl-modules-5.26 (5.26.1-6ubuntu0.5) over (5.26.1-6ubuntu0.3) ... Preparing to unpack .../tar_1.29b-2ubuntu0.2_amd64.deb ... Unpacking tar (1.29b-2ubuntu0.2) over (1.29b-2ubuntu0.1) ... Setting up tar (1.29b-2ubuntu0.2) ... update-alternatives: warning: forcing reinstallation of alternative /usr/sbin/rmt-tar because link group rmt is broken (Reading database ... 160983 files and directories currently installed.) Preparing to unpack .../libc-bin_2.27-3ubuntu1.4_amd64.deb ... Unpacking libc-bin (2.27-3ubuntu1.4) over (2.27-3ubuntu1.2) ... Setting up libc-bin (2.27-3ubuntu1.4) ... /sbin/ldconfig.real: /usr/local/lib/python3.7/dist-packages/ideep4py/lib/libmkldnn.so.0 is not a symbolic link (Reading database ... 160983 files and directories currently installed.) Preparing to unpack .../libaudit-common_1%3a2.8.2-1ubuntu1.1_all.deb ... Unpacking libaudit-common (1:2.8.2-1ubuntu1.1) over (1:2.8.2-1ubuntu1) ... Setting up libaudit-common (1:2.8.2-1ubuntu1.1) ... (Reading database ... 160983 files and directories currently installed.) Preparing to unpack .../libaudit1_1%3a2.8.2-1ubuntu1.1_amd64.deb ... Unpacking libaudit1:amd64 (1:2.8.2-1ubuntu1.1) over (1:2.8.2-1ubuntu1) ... Setting up libaudit1:amd64 (1:2.8.2-1ubuntu1.1) ... (Reading database ... 160983 files and directories currently installed.) Preparing to unpack .../libzstd1_1.3.3+dfsg-2ubuntu1.2_amd64.deb ... Unpacking libzstd1:amd64 (1.3.3+dfsg-2ubuntu1.2) over (1.3.3+dfsg-2ubuntu1.1) ... Setting up libzstd1:amd64 (1.3.3+dfsg-2ubuntu1.2) ... (Reading database ... 160983 files and directories currently installed.) Preparing to unpack .../libnettle6_3.4-1ubuntu0.1_amd64.deb ... Unpacking libnettle6:amd64 (3.4-1ubuntu0.1) over (3.4-1) ... Setting up libnettle6:amd64 (3.4-1ubuntu0.1) ... (Reading database ... 160983 files and directories currently installed.) Preparing to unpack .../libhogweed4_3.4-1ubuntu0.1_amd64.deb ... Unpacking libhogweed4:amd64 (3.4-1ubuntu0.1) over (3.4-1) ... Setting up libhogweed4:amd64 (3.4-1ubuntu0.1) ... (Reading database ... 160983 files and directories currently installed.) Preparing to unpack .../libp11-kit0_0.23.9-2ubuntu0.1_amd64.deb ... Unpacking libp11-kit0:amd64 (0.23.9-2ubuntu0.1) over (0.23.9-2) ... Setting up libp11-kit0:amd64 (0.23.9-2ubuntu0.1) ... (Reading database ... 160983 files and directories currently installed.) Preparing to unpack .../libseccomp2_2.5.1-1ubuntu1~18.04.1_amd64.deb ... Unpacking libseccomp2:amd64 (2.5.1-1ubuntu1~18.04.1) over (2.4.3-1ubuntu3.18.04.3) ... Setting up libseccomp2:amd64 (2.5.1-1ubuntu1~18.04.1) ... (Reading database ... 160983 files and directories currently installed.) Preparing to unpack .../ubuntu-keyring_2018.09.18.1~18.04.2_all.deb ... Unpacking ubuntu-keyring (2018.09.18.1~18.04.2) over (2018.09.18.1~18.04.0) ... Setting up ubuntu-keyring (2018.09.18.1~18.04.2) ... (Reading database ... 160983 files and directories currently installed.) Preparing to unpack .../00-openssl_1.1.1-1ubuntu2.1~18.04.9_amd64.deb ... Unpacking openssl (1.1.1-1ubuntu2.1~18.04.9) over (1.1.1-1ubuntu2.1~18.04.6) ... Preparing to unpack .../01-binutils-x86-64-linux-gnu_2.30-21ubuntu1~18.04.5_amd64.deb ... Unpacking binutils-x86-64-linux-gnu (2.30-21ubuntu1~18.04.5) over (2.30-21ubuntu1~18.04.4) ... Preparing to unpack .../02-binutils-common_2.30-21ubuntu1~18.04.5_amd64.deb ... Unpacking binutils-common:amd64 (2.30-21ubuntu1~18.04.5) over (2.30-21ubuntu1~18.04.4) ... Preparing to unpack .../03-binutils_2.30-21ubuntu1~18.04.5_amd64.deb ... Unpacking binutils (2.30-21ubuntu1~18.04.5) over (2.30-21ubuntu1~18.04.4) ... Preparing to unpack .../04-libbinutils_2.30-21ubuntu1~18.04.5_amd64.deb ... Unpacking libbinutils:amd64 (2.30-21ubuntu1~18.04.5) over (2.30-21ubuntu1~18.04.4) ... Preparing to unpack .../05-cuda-compat-11-0_450.102.04-1_amd64.deb ... Unpacking cuda-compat-11-0 (450.102.04-1) over (450.80.02-1) ... Preparing to unpack .../06-libcublas10_10.2.3.254-1_amd64.deb ... Unpacking libcublas10 (10.2.3.254-1) over (10.2.1.243-1) ... Preparing to unpack .../07-libcublas-dev_10.2.3.254-1_amd64.deb ... Unpacking libcublas-dev (10.2.3.254-1) over (10.2.1.243-1) ... Preparing to unpack .../08-libcudnn7-dev_7.6.5.32-1+cuda10.2_amd64.deb ... update-alternatives: removing manually selected alternative - switching libcudnn to auto mode update-alternatives: using /usr/include/x86_64-linux-gnu/cudnn_v8.h to provide /usr/include/cudnn.h (libcudnn) in auto mode Unpacking libcudnn7-dev (7.6.5.32-1+cuda10.2) over (7.6.5.32-1+cuda10.1) ... Preparing to unpack .../09-libcudnn7_7.6.5.32-1+cuda10.2_amd64.deb ... Unpacking libcudnn7 (7.6.5.32-1+cuda10.2) over (7.6.5.32-1+cuda10.1) ... Selecting previously unselected package libgeos-3.8.0:amd64. Preparing to unpack .../10-libgeos-3.8.0_3.8.0-1~bionic0_amd64.deb ... Unpacking libgeos-3.8.0:amd64 (3.8.0-1~bionic0) ... Preparing to unpack .../11-libgeos-dev_3.8.0-1~bionic0_amd64.deb ... Unpacking libgeos-dev (3.8.0-1~bionic0) over (3.6.2-1build2) ... Preparing to unpack .../12-libgeos-c1v5_3.8.0-1~bionic0_amd64.deb ... Unpacking libgeos-c1v5:amd64 (3.8.0-1~bionic0) over (3.6.2-1build2) ... Preparing to unpack .../13-proj-data_7.0.0-1~bionic0_all.deb ... Unpacking proj-data (7.0.0-1~bionic0) over (4.9.3-2) ... Selecting previously unselected package libproj19:amd64. Preparing to unpack .../14-libproj19_7.0.0-1~bionic0_amd64.deb ... Unpacking libproj19:amd64 (7.0.0-1~bionic0) ... Selecting previously unselected package libgeotiff5:amd64. Preparing to unpack .../15-libgeotiff5_1.5.1-2~bionic2_amd64.deb ... Unpacking libgeotiff5:amd64 (1.5.1-2~bionic2) ... Preparing to unpack .../16-libproj-dev_7.0.0-1~bionic0_amd64.deb ... Unpacking libproj-dev:amd64 (7.0.0-1~bionic0) over (4.9.3-2) ... Preparing to unpack .../17-libgeotiff-dev_1.5.1-2~bionic2_amd64.deb ... Unpacking libgeotiff-dev:amd64 (1.5.1-2~bionic2) over (1.4.2-2build1) ... Selecting previously unselected package libproj13:amd64. Preparing to unpack .../18-libproj13_5.2.0-1~bionic0_amd64.deb ... Unpacking libproj13:amd64 (5.2.0-1~bionic0) ... Preparing to unpack .../19-libgeotiff2_1.4.2-2build1+bionic0_amd64.deb ... Unpacking libgeotiff2:amd64 (1.4.2-2build1+bionic0) over (1.4.2-2build1) ... Preparing to unpack .../20-libsasl2-modules-db_2.1.27~101-g0780600+dfsg-3ubuntu2.3_amd64.deb ... Unpacking libsasl2-modules-db:amd64 (2.1.27~101-g0780600+dfsg-3ubuntu2.3) over (2.1.27~101-g0780600+dfsg-3ubuntu2.1) ... Preparing to unpack .../21-libsasl2-2_2.1.27~101-g0780600+dfsg-3ubuntu2.3_amd64.deb ... Unpacking libsasl2-2:amd64 (2.1.27~101-g0780600+dfsg-3ubuntu2.3) over (2.1.27~101-g0780600+dfsg-3ubuntu2.1) ... Preparing to unpack .../22-libldap-common_2.4.45+dfsg-1ubuntu1.10_all.deb ... Unpacking libldap-common (2.4.45+dfsg-1ubuntu1.10) over (2.4.45+dfsg-1ubuntu1.6) ... Preparing to unpack .../23-libldap-2.4-2_2.4.45+dfsg-1ubuntu1.10_amd64.deb ... Unpacking libldap-2.4-2:amd64 (2.4.45+dfsg-1ubuntu1.10) over (2.4.45+dfsg-1ubuntu1.6) ... Preparing to unpack .../24-libspatialite-dev_4.3.0a-6~bionic3_amd64.deb ... Unpacking libspatialite-dev:amd64 (4.3.0a-6~bionic3) over (4.3.0a-5build1) ... Preparing to unpack .../25-libspatialite7_4.3.0a-6~bionic3_amd64.deb ... Unpacking libspatialite7:amd64 (4.3.0a-6~bionic3) over (4.3.0a-5build1) ... Selecting previously unselected package linux-headers-4.15.0-141. Preparing to unpack .../26-linux-headers-4.15.0-141_4.15.0-141.145_all.deb ... Unpacking linux-headers-4.15.0-141 (4.15.0-141.145) ... Selecting previously unselected package linux-headers-4.15.0-141-generic. Preparing to unpack .../27-linux-headers-4.15.0-141-generic_4.15.0-141.145_amd64.deb ... Unpacking linux-headers-4.15.0-141-generic (4.15.0-141.145) ... Preparing to unpack .../28-linux-headers-generic_4.15.0.141.128_amd64.deb ... Unpacking linux-headers-generic (4.15.0.141.128) over (4.15.0.140.127) ... Preparing to unpack .../29-linux-libc-dev_4.15.0-141.145_amd64.deb ... Unpacking linux-libc-dev:amd64 (4.15.0-141.145) over (4.15.0-118.119) ... Selecting previously unselected package proj-bin. Preparing to unpack .../30-proj-bin_7.0.0-1~bionic0_amd64.deb ... Unpacking proj-bin (7.0.0-1~bionic0) ... Preparing to unpack .../31-r-cran-dbplyr_2.1.1-1cran1.1804.0_all.deb ... Unpacking r-cran-dbplyr (2.1.1-1cran1.1804.0) over (2.1.0-1cran1.1804.0) ... Preparing to unpack .../32-r-cran-pkgload_1.2.1-1cran1.1804.0_all.deb ... Unpacking r-cran-pkgload (1.2.1-1cran1.1804.0) over (1.2.0-1cran1.1804.0) ... Preparing to unpack .../33-r-cran-devtools_2.4.0-1cran1.1804.0_all.deb ... Unpacking r-cran-devtools (2.4.0-1cran1.1804.0) over (2.3.2-1cran1.1804.0) ... Preparing to unpack .../34-xserver-common_2%3a1.19.6-1ubuntu4.9_all.deb ... Unpacking xserver-common (2:1.19.6-1ubuntu4.9) over (2:1.19.6-1ubuntu4.8) ... Preparing to unpack .../35-xserver-xorg-core-hwe-18.04_2%3a1.20.8-2ubuntu2.2~18.04.5_amd64.deb ... Unpacking xserver-xorg-core-hwe-18.04 (2:1.20.8-2ubuntu2.2~18.04.5) over (2:1.20.8-2ubuntu2.2~18.04.4) ... Preparing to unpack .../36-gnupg2_2.2.4-1ubuntu1.4_all.deb ... Unpacking gnupg2 (2.2.4-1ubuntu1.4) over (2.2.4-1ubuntu1.3) ... Setting up libcudnn7 (7.6.5.32-1+cuda10.2) ... Setting up libgeos-3.8.0:amd64 (3.8.0-1~bionic0) ... Setting up linux-headers-4.15.0-141 (4.15.0-141.145) ... Setting up xserver-common (2:1.19.6-1ubuntu4.9) ... Setting up r-cran-pkgload (1.2.1-1cran1.1804.0) ... Setting up libldap-common (2.4.45+dfsg-1ubuntu1.10) ... Setting up r-cran-devtools (2.4.0-1cran1.1804.0) ... Setting up cuda-compat-11-0 (450.102.04-1) ... Setting up libsasl2-modules-db:amd64 (2.1.27~101-g0780600+dfsg-3ubuntu2.3) ... Setting up linux-libc-dev:amd64 (4.15.0-141.145) ... Setting up libsasl2-2:amd64 (2.1.27~101-g0780600+dfsg-3ubuntu2.3) ... Setting up libcudnn7-dev (7.6.5.32-1+cuda10.2) ... update-alternatives: using /usr/include/x86_64-linux-gnu/cudnn_v7.h to provide /usr/include/cudnn.h (libcudnn) in manual mode Setting up perl-modules-5.26 (5.26.1-6ubuntu0.5) ... Setting up binutils-common:amd64 (2.30-21ubuntu1~18.04.5) ... Setting up gnupg2 (2.2.4-1ubuntu1.4) ... Setting up libgeos-c1v5:amd64 (3.8.0-1~bionic0) ... Setting up libcublas10 (10.2.3.254-1) ... Setting up libcublas-dev (10.2.3.254-1) ... Setting up libperl5.26:amd64 (5.26.1-6ubuntu0.5) ... Setting up libldap-2.4-2:amd64 (2.4.45+dfsg-1ubuntu1.10) ... Setting up openssl (1.1.1-1ubuntu2.1~18.04.9) ... Setting up r-cran-dbplyr (2.1.1-1cran1.1804.0) ... Setting up xserver-xorg-core-hwe-18.04 (2:1.20.8-2ubuntu2.2~18.04.5) ... Setting up linux-headers-4.15.0-141-generic (4.15.0-141.145) ... /etc/kernel/header_postinst.d/dkms: * dkms: running auto installation service for kernel 4.15.0-141-generic Kernel preparation unnecessary for this kernel. Skipping... applying patch disable_fstack-clash-protection_fcf-protection.patch...patching file Kbuild Hunk #1 succeeded at 85 (offset 14 lines). Building module: cleaning build area...(bad exit status: 2) unset ARCH; [ ! -h /usr/bin/cc ] && export CC=/usr/bin/gcc; env NV_VERBOSE=1 'make' -j2 NV_EXCLUDE_BUILD_MODULES='' KERNEL_UNAME=4.15.0-141-generic IGNORE_XEN_PRESENCE=1 IGNORE_CC_MISMATCH=1 SYSSRC=/lib/modules/4.15.0-141-generic/build LD=/usr/bin/ld.bfd modules........................................ cleaning build area...(bad exit status: 2) DKMS: build completed. nvidia.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/4.15.0-141-generic/updates/dkms/ nvidia-modeset.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/4.15.0-141-generic/updates/dkms/ nvidia-drm.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/4.15.0-141-generic/updates/dkms/ nvidia-uvm.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/4.15.0-141-generic/updates/dkms/ depmod... DKMS: install completed. ...done. Setting up proj-data (7.0.0-1~bionic0) ... Setting up linux-headers-generic (4.15.0.141.128) ... Setting up libbinutils:amd64 (2.30-21ubuntu1~18.04.5) ... Setting up libproj19:amd64 (7.0.0-1~bionic0) ... Setting up libproj13:amd64 (5.2.0-1~bionic0) ... Setting up libgeos-dev (3.8.0-1~bionic0) ... Setting up libgeotiff5:amd64 (1.5.1-2~bionic2) ... Setting up perl (5.26.1-6ubuntu0.5) ... Setting up proj-bin (7.0.0-1~bionic0) ... Setting up libgeotiff2:amd64 (1.4.2-2build1+bionic0) ... Setting up libspatialite7:amd64 (4.3.0a-6~bionic3) ... Setting up libspatialite-dev:amd64 (4.3.0a-6~bionic3) ... Setting up binutils-x86-64-linux-gnu (2.30-21ubuntu1~18.04.5) ... Setting up libproj-dev:amd64 (7.0.0-1~bionic0) ... Setting up binutils (2.30-21ubuntu1~18.04.5) ... Setting up libgeotiff-dev:amd64 (1.5.1-2~bionic2) ... Processing triggers for libc-bin (2.27-3ubuntu1.4) ... /sbin/ldconfig.real: /usr/local/lib/python3.7/dist-packages/ideep4py/lib/libmkldnn.so.0 is not a symbolic link Processing triggers for man-db (2.8.3-2ubuntu0.1) ... Processing triggers for mime-support (3.60ubuntu1) ...
!apt-get install gdal-bin
Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libgeos-3.6.2 libgeotiff2 libproj12 libproj13 linux-headers-4.15.0-140 linux-headers-4.15.0-140-generic python-numpy r-cran-covr r-cran-crosstalk r-cran-dt r-cran-htmlwidgets r-cran-later r-cran-lazyeval r-cran-promises r-cran-rex Use 'apt autoremove' to remove them. The following additional packages will be installed: gdal-data libcfitsio-dev libcfitsio5 libcharls-dev libgdal-dev libgdal26 libogdi-dev libogdi4.1 libopencv-calib3d-dev libopencv-calib3d3.2 libopencv-contrib-dev libopencv-contrib3.2 libopencv-core-dev libopencv-core3.2 libopencv-dev libopencv-features2d-dev libopencv-features2d3.2 libopencv-flann-dev libopencv-flann3.2 libopencv-highgui-dev libopencv-highgui3.2 libopencv-imgcodecs-dev libopencv-imgcodecs3.2 libopencv-imgproc-dev libopencv-imgproc3.2 libopencv-ml-dev libopencv-ml3.2 libopencv-objdetect-dev libopencv-objdetect3.2 libopencv-photo-dev libopencv-photo3.2 libopencv-shape-dev libopencv-shape3.2 libopencv-stitching-dev libopencv-stitching3.2 libopencv-superres-dev libopencv-superres3.2 libopencv-ts-dev libopencv-video-dev libopencv-video3.2 libopencv-videoio-dev libopencv-videoio3.2 libopencv-videostab-dev libopencv-videostab3.2 libopencv-viz-dev libopencv-viz3.2 libopencv3.2-java libopencv3.2-jni libvtk6.3 libzstd-dev python3-gdal python3-numpy Suggested packages: libgdal-grass libgdal-doc ogdi-bin opencv-doc vtk6-doc vtk6-examples python-numpy-doc python3-nose python3-numpy-dbg Recommended packages: libcfitsio-doc opencv-data The following packages will be REMOVED: libgdal20 libogdi3.2 libogdi3.2-dev python-gdal The following NEW packages will be installed: libcfitsio-dev libcfitsio5 libcharls-dev libgdal26 libogdi-dev libogdi4.1 libzstd-dev python3-gdal python3-numpy The following packages will be upgraded: gdal-bin gdal-data libgdal-dev libopencv-calib3d-dev libopencv-calib3d3.2 libopencv-contrib-dev libopencv-contrib3.2 libopencv-core-dev libopencv-core3.2 libopencv-dev libopencv-features2d-dev libopencv-features2d3.2 libopencv-flann-dev libopencv-flann3.2 libopencv-highgui-dev libopencv-highgui3.2 libopencv-imgcodecs-dev libopencv-imgcodecs3.2 libopencv-imgproc-dev libopencv-imgproc3.2 libopencv-ml-dev libopencv-ml3.2 libopencv-objdetect-dev libopencv-objdetect3.2 libopencv-photo-dev libopencv-photo3.2 libopencv-shape-dev libopencv-shape3.2 libopencv-stitching-dev libopencv-stitching3.2 libopencv-superres-dev libopencv-superres3.2 libopencv-ts-dev libopencv-video-dev libopencv-video3.2 libopencv-videoio-dev libopencv-videoio3.2 libopencv-videostab-dev libopencv-videostab3.2 libopencv-viz-dev libopencv-viz3.2 libopencv3.2-java libopencv3.2-jni libvtk6.3 44 upgraded, 9 newly installed, 4 to remove and 4 not upgraded. Need to get 63.7 MB of archives. After this operation, 25.1 MB of additional disk space will be used. Get:1 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [237 kB] Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libcfitsio5 amd64 3.430-2 [446 kB] Get:3 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-ts-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [279 kB] Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libcfitsio-dev amd64 3.430-2 [494 kB] Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libcharls-dev amd64 1.1.0+dfsg-2 [20.4 kB] Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libzstd-dev amd64 1.3.3+dfsg-2ubuntu1.2 [230 kB] Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] Get:8 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-contrib-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [1,877 kB] Get:9 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-videostab-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [135 kB] Get:10 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-stitching-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [222 kB] Get:11 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-calib3d-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [516 kB] Get:12 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-features2d-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [293 kB] Get:13 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-flann-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [176 kB] Get:14 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-contrib3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [1,464 kB] Get:15 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv3.2-jni amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [201 kB] Get:16 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-videostab3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [111 kB] Get:17 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-stitching3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [180 kB] Get:18 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-calib3d3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [440 kB] Get:19 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-features2d3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [244 kB] Get:20 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-flann3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [105 kB] Get:21 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-objdetect-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [176 kB] Get:22 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-objdetect3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [151 kB] Get:23 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-ml-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [296 kB] Get:24 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-ml3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [236 kB] Get:25 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-superres-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [65.9 kB] Get:26 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-highgui-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [44.2 kB] Get:27 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-videoio-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [124 kB] Get:28 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-imgcodecs-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [132 kB] Get:29 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-superres3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [56.0 kB] Get:30 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-highgui3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [32.4 kB] Get:31 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-videoio3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [90.9 kB] Get:32 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-viz-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [163 kB] Get:33 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-shape-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [84.0 kB] Get:34 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-video-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [162 kB] Get:35 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-photo-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [238 kB] Get:36 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-imgproc-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [1,023 kB] Get:37 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-core-dev amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [1,109 kB] Get:38 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libgdal-dev amd64 3.0.4+dfsg-1~bionic0 [7,659 kB] Get:39 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-imgcodecs3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [95.8 kB] Get:40 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libvtk6.3 amd64 6.3.0+dfsg2-2build4~bionic4 [31.5 MB] Get:41 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 gdal-bin amd64 3.0.4+dfsg-1~bionic0 [500 kB] Get:42 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 gdal-data all 3.0.4+dfsg-1~bionic0 [430 kB] Get:43 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libogdi-dev amd64 4.1.0+ds-1~bionic2 [25.4 kB] Get:44 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libogdi4.1 amd64 4.1.0+ds-1~bionic2 [200 kB] Get:45 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libgdal26 amd64 3.0.4+dfsg-1~bionic0 [6,164 kB] Get:46 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-viz3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [124 kB] Get:47 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-shape3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [69.7 kB] Get:48 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-video3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [136 kB] Get:49 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-photo3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [203 kB] Get:50 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-imgproc3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [831 kB] Get:51 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv-core3.2 amd64 3.2.0+dfsg-4ubuntu0.1+bionic4 [720 kB] Get:52 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 libopencv3.2-java all 3.2.0+dfsg-4ubuntu0.1+bionic4 [401 kB] Get:53 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic/main amd64 python3-gdal amd64 3.0.4+dfsg-1~bionic0 [760 kB] Fetched 63.7 MB in 28s (2,273 kB/s) Extracting templates from packages: 100% (Reading database ... 190365 files and directories currently installed.) Removing python-gdal (2.2.3+dfsg-2) ... (Reading database ... 190255 files and directories currently installed.) Preparing to unpack .../00-libopencv-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-dev (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../01-libopencv-ts-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-ts-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../02-libopencv-contrib-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-contrib-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../03-libopencv-videostab-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-videostab-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../04-libopencv-stitching-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-stitching-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../05-libopencv-calib3d-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-calib3d-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../06-libopencv-features2d-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-features2d-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../07-libopencv-flann-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-flann-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../08-libopencv-contrib3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-contrib3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../09-libopencv3.2-jni_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv3.2-jni (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../10-libopencv-videostab3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-videostab3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../11-libopencv-stitching3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-stitching3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../12-libopencv-calib3d3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-calib3d3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../13-libopencv-features2d3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-features2d3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../14-libopencv-flann3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-flann3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../15-libopencv-objdetect-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-objdetect-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../16-libopencv-objdetect3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-objdetect3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../17-libopencv-ml-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-ml-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../18-libopencv-ml3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-ml3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../19-libopencv-superres-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-superres-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../20-libopencv-highgui-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-highgui-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../21-libopencv-videoio-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-videoio-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../22-libopencv-imgcodecs-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-imgcodecs-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../23-libopencv-superres3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-superres3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../24-libopencv-highgui3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-highgui3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../25-libopencv-videoio3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-videoio3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../26-libopencv-viz-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-viz-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../27-libopencv-shape-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-shape-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../28-libopencv-video-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-video-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../29-libopencv-photo-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-photo-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../30-libopencv-imgproc-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-imgproc-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../31-libopencv-core-dev_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-core-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../32-libgdal-dev_3.0.4+dfsg-1~bionic0_amd64.deb ... Unpacking libgdal-dev (3.0.4+dfsg-1~bionic0) over (2.2.3+dfsg-2) ... Preparing to unpack .../33-libopencv-imgcodecs3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-imgcodecs3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../34-libvtk6.3_6.3.0+dfsg2-2build4~bionic4_amd64.deb ... Unpacking libvtk6.3 (6.3.0+dfsg2-2build4~bionic4) over (6.3.0+dfsg1-11build1) ... Preparing to unpack .../35-gdal-bin_3.0.4+dfsg-1~bionic0_amd64.deb ... Unpacking gdal-bin (3.0.4+dfsg-1~bionic0) over (2.2.3+dfsg-2) ... (Reading database ... 190288 files and directories currently installed.) Removing libgdal20 (2.2.3+dfsg-2) ... (Reading database ... 190281 files and directories currently installed.) Preparing to unpack .../gdal-data_3.0.4+dfsg-1~bionic0_all.deb ... Unpacking gdal-data (3.0.4+dfsg-1~bionic0) over (2.2.3+dfsg-2) ... Selecting previously unselected package libcfitsio5:amd64. Preparing to unpack .../libcfitsio5_3.430-2_amd64.deb ... Unpacking libcfitsio5:amd64 (3.430-2) ... Selecting previously unselected package libcfitsio-dev:amd64. Preparing to unpack .../libcfitsio-dev_3.430-2_amd64.deb ... Unpacking libcfitsio-dev:amd64 (3.430-2) ... Selecting previously unselected package libcharls-dev:amd64. Preparing to unpack .../libcharls-dev_1.1.0+dfsg-2_amd64.deb ... Unpacking libcharls-dev:amd64 (1.1.0+dfsg-2) ... (Reading database ... 190310 files and directories currently installed.) Removing libogdi3.2-dev (3.2.0+ds-2) ... Selecting previously unselected package libogdi-dev. (Reading database ... 190297 files and directories currently installed.) Preparing to unpack .../libogdi-dev_4.1.0+ds-1~bionic2_amd64.deb ... Unpacking libogdi-dev (4.1.0+ds-1~bionic2) ... Selecting previously unselected package libzstd-dev:amd64. Preparing to unpack .../libzstd-dev_1.3.3+dfsg-2ubuntu1.2_amd64.deb ... Unpacking libzstd-dev:amd64 (1.3.3+dfsg-2ubuntu1.2) ... (Reading database ... 190331 files and directories currently installed.) Removing libogdi3.2 (3.2.0+ds-2) ... Selecting previously unselected package libogdi4.1. (Reading database ... 190313 files and directories currently installed.) Preparing to unpack .../00-libogdi4.1_4.1.0+ds-1~bionic2_amd64.deb ... Unpacking libogdi4.1 (4.1.0+ds-1~bionic2) ... Selecting previously unselected package libgdal26. Preparing to unpack .../01-libgdal26_3.0.4+dfsg-1~bionic0_amd64.deb ... Unpacking libgdal26 (3.0.4+dfsg-1~bionic0) ... Preparing to unpack .../02-libopencv-viz3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-viz3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../03-libopencv-shape3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-shape3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../04-libopencv-video3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-video3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../05-libopencv-photo3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-photo3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../06-libopencv-imgproc3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-imgproc3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../07-libopencv-core3.2_3.2.0+dfsg-4ubuntu0.1+bionic4_amd64.deb ... Unpacking libopencv-core3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Preparing to unpack .../08-libopencv3.2-java_3.2.0+dfsg-4ubuntu0.1+bionic4_all.deb ... Unpacking libopencv3.2-java (3.2.0+dfsg-4ubuntu0.1+bionic4) over (3.2.0+dfsg-4ubuntu0.1) ... Selecting previously unselected package python3-numpy. Preparing to unpack .../09-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... Selecting previously unselected package python3-gdal. Preparing to unpack .../10-python3-gdal_3.0.4+dfsg-1~bionic0_amd64.deb ... Unpacking python3-gdal (3.0.4+dfsg-1~bionic0) ... Setting up libcfitsio5:amd64 (3.430-2) ... Setting up libcfitsio-dev:amd64 (3.430-2) ... Setting up python3-numpy (1:1.13.3-2ubuntu1) ... Setting up libopencv-core3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up gdal-data (3.0.4+dfsg-1~bionic0) ... Setting up libcharls-dev:amd64 (1.1.0+dfsg-2) ... Setting up libopencv-core-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libzstd-dev:amd64 (1.3.3+dfsg-2ubuntu1.2) ... Setting up libogdi4.1 (4.1.0+ds-1~bionic2) ... Setting up libopencv-ml3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-ml-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-imgproc3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-flann3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-video3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-imgproc-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-photo3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-ts-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libgdal26 (3.0.4+dfsg-1~bionic0) ... Setting up libogdi-dev (4.1.0+ds-1~bionic2) ... Setting up libopencv-photo-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-flann-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-imgcodecs3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up python3-gdal (3.0.4+dfsg-1~bionic0) ... Setting up libvtk6.3 (6.3.0+dfsg2-2build4~bionic4) ... Setting up libopencv-shape3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-video-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up gdal-bin (3.0.4+dfsg-1~bionic0) ... Setting up libopencv-videoio3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-shape-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libgdal-dev (3.0.4+dfsg-1~bionic0) ... Setting up libopencv-imgcodecs-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-viz3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-superres3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-highgui3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-videoio-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-viz-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-objdetect3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-highgui-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-features2d3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-superres-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-features2d-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-calib3d3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-stitching3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-calib3d-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-objdetect-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-videostab3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-stitching-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-contrib3.2:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-videostab-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-contrib-dev:amd64 (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv3.2-jni (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv3.2-java (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Setting up libopencv-dev (3.2.0+dfsg-4ubuntu0.1+bionic4) ... Processing triggers for man-db (2.8.3-2ubuntu0.1) ... Processing triggers for libc-bin (2.27-3ubuntu1.4) ... /sbin/ldconfig.real: /usr/local/lib/python3.7/dist-packages/ideep4py/lib/libmkldnn.so.0 is not a symbolic link
!ogrinfo --version
GDAL 3.0.4, released 2020/01/28
pip install 'gdal==3.0.4'
Collecting gdal==3.0.4
Downloading GDAL-3.0.4.tar.gz (577 kB)
|████████████████████████████████| 577 kB 3.9 MB/s
Building wheels for collected packages: gdal
Building wheel for gdal (setup.py) ... done
Created wheel for gdal: filename=GDAL-3.0.4-cp37-cp37m-linux_x86_64.whl size=2065914 sha256=ee5b24085a136ef8d8f3bd252f579142c97bd41427dba0477416fd7729c8f238
Stored in directory: /root/.cache/pip/wheels/fb/6c/d5/db1e3aade3b44a6c9c09df7ab991988e63d6b0b49b26e8d540
Successfully built gdal
Installing collected packages: gdal
Attempting uninstall: gdal
Found existing installation: GDAL 2.2.2
Uninstalling GDAL-2.2.2:
Successfully uninstalled GDAL-2.2.2
Successfully installed gdal-3.0.4
#!wget -c https://geoftp.ibge.gov.br/organizacao_do_territorio/malhas_territoriais/malhas_municipais/municipio_2019/Brasil/BR/br_unidades_da_federacao.zip
!wget -c https://www.dropbox.com/s/sgalgfxonbty2z6/ne_10m_admin_1_states_provinces.zip
# !wget -c https://www.dropbox.com/s/514312gwhstn5r9/br_unidades_da_federacao.zip
!unzip ne_10m_admin_1_states_provinces.zip
# !unzip br_unidades_da_federacao.zip
--2021-04-14 14:43:57-- https://www.dropbox.com/s/sgalgfxonbty2z6/ne_10m_admin_1_states_provinces.zip Resolving www.dropbox.com (www.dropbox.com)... 162.125.5.18, 2620:100:601d:18::a27d:512 Connecting to www.dropbox.com (www.dropbox.com)|162.125.5.18|:443... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: /s/raw/sgalgfxonbty2z6/ne_10m_admin_1_states_provinces.zip [following] --2021-04-14 14:43:57-- https://www.dropbox.com/s/raw/sgalgfxonbty2z6/ne_10m_admin_1_states_provinces.zip Reusing existing connection to www.dropbox.com:443. HTTP request sent, awaiting response... 302 Found Location: https://uc8f34a19bb6793138f9640dd7f3.dl.dropboxusercontent.com/cd/0/inline/BMklbMovkGnZV11o6TWiNa-BXzcWvc8Yd7IuRtYWKmk6nhGuDj4H7vrIzU-xefzwXfjacwXJ8qrlI5SN6Gy_GSN58qxX0GM0FslonmtN54Jsycsy7Hnk7UnbhafgXjo_BBTb_gIvyNT5hT2qD0t5mGBU/file# [following] --2021-04-14 14:43:57-- https://uc8f34a19bb6793138f9640dd7f3.dl.dropboxusercontent.com/cd/0/inline/BMklbMovkGnZV11o6TWiNa-BXzcWvc8Yd7IuRtYWKmk6nhGuDj4H7vrIzU-xefzwXfjacwXJ8qrlI5SN6Gy_GSN58qxX0GM0FslonmtN54Jsycsy7Hnk7UnbhafgXjo_BBTb_gIvyNT5hT2qD0t5mGBU/file Resolving uc8f34a19bb6793138f9640dd7f3.dl.dropboxusercontent.com (uc8f34a19bb6793138f9640dd7f3.dl.dropboxusercontent.com)... 162.125.5.15, 2620:100:601d:15::a27d:50f Connecting to uc8f34a19bb6793138f9640dd7f3.dl.dropboxusercontent.com (uc8f34a19bb6793138f9640dd7f3.dl.dropboxusercontent.com)|162.125.5.15|:443... connected. HTTP request sent, awaiting response... 302 Found Location: /cd/0/inline2/BMmsGwPG90eglXOSGXEvgAaaCUgpEu11tSQ77bDWrwNw-5BO6gRp1qXq_HOXQc_a3L0BQaNZtLi9dPdQ3h5WUGyQTic1E9-KKGYzdm2EJSnU2tQ9AnrPafJzzXqO60pNxwwVswndJPqmjNI2CUDsf4QI-snUhmxmSgYTmn3o2y0eFSlTmPiVkSQaGN-bryjCun9BZubL3sJrcim4fpnjiRQSbm-yzCgbWIWvxtfbD3em2TZxs4NaUqXG6sJF-BqNBYKnlJ-s-T75rzcQpPKSbySOsJlkAnLZOWLl1vPAKuLPhk2we2eobiLupDmyGJiLtDn8dpPzOWgsBD6cotGrnPLT9jOa2XmXzz-WcCCWV-wZuF4ihCuFhslugxTDcD4bKg4/file [following] --2021-04-14 14:43:58-- https://uc8f34a19bb6793138f9640dd7f3.dl.dropboxusercontent.com/cd/0/inline2/BMmsGwPG90eglXOSGXEvgAaaCUgpEu11tSQ77bDWrwNw-5BO6gRp1qXq_HOXQc_a3L0BQaNZtLi9dPdQ3h5WUGyQTic1E9-KKGYzdm2EJSnU2tQ9AnrPafJzzXqO60pNxwwVswndJPqmjNI2CUDsf4QI-snUhmxmSgYTmn3o2y0eFSlTmPiVkSQaGN-bryjCun9BZubL3sJrcim4fpnjiRQSbm-yzCgbWIWvxtfbD3em2TZxs4NaUqXG6sJF-BqNBYKnlJ-s-T75rzcQpPKSbySOsJlkAnLZOWLl1vPAKuLPhk2we2eobiLupDmyGJiLtDn8dpPzOWgsBD6cotGrnPLT9jOa2XmXzz-WcCCWV-wZuF4ihCuFhslugxTDcD4bKg4/file Reusing existing connection to uc8f34a19bb6793138f9640dd7f3.dl.dropboxusercontent.com:443. HTTP request sent, awaiting response... 200 OK Length: 14269447 (14M) [application/zip] Saving to: ‘ne_10m_admin_1_states_provinces.zip’ ne_10m_admin_1_stat 100%[===================>] 13.61M 31.9MB/s in 0.4s 2021-04-14 14:43:59 (31.9 MB/s) - ‘ne_10m_admin_1_states_provinces.zip’ saved [14269447/14269447] Archive: ne_10m_admin_1_states_provinces.zip inflating: ne_10m_admin_1_states_provinces.shx inflating: ne_10m_admin_1_states_provinces.dbf inflating: ne_10m_admin_1_states_provinces.shp
!wget -c https://www.dropbox.com/s/fdgnaqt91cy3x97/IR4AVHRR6.cpt
--2021-04-14 14:43:59-- https://www.dropbox.com/s/fdgnaqt91cy3x97/IR4AVHRR6.cpt Resolving www.dropbox.com (www.dropbox.com)... 162.125.5.18, 2620:100:601d:18::a27d:512 Connecting to www.dropbox.com (www.dropbox.com)|162.125.5.18|:443... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: /s/raw/fdgnaqt91cy3x97/IR4AVHRR6.cpt [following] --2021-04-14 14:44:00-- https://www.dropbox.com/s/raw/fdgnaqt91cy3x97/IR4AVHRR6.cpt Reusing existing connection to www.dropbox.com:443. HTTP request sent, awaiting response... 302 Found Location: https://uc274ec77fe3055a2ffd2a56bf7e.dl.dropboxusercontent.com/cd/0/inline/BMmAE-ZO1R48-7p9zkMAODk_uk0nIS0Hkug7WQu3F1q7RlONMpSso0fbYUFY01h76HHhUO-wjZwUiC25wmmG9iachbT32wl9IoqKrgkMNpB_cA6ndvgLM8TmSVFMreQ2m1jVjeoOm-j3VJp6DO1an6-p/file# [following] --2021-04-14 14:44:00-- https://uc274ec77fe3055a2ffd2a56bf7e.dl.dropboxusercontent.com/cd/0/inline/BMmAE-ZO1R48-7p9zkMAODk_uk0nIS0Hkug7WQu3F1q7RlONMpSso0fbYUFY01h76HHhUO-wjZwUiC25wmmG9iachbT32wl9IoqKrgkMNpB_cA6ndvgLM8TmSVFMreQ2m1jVjeoOm-j3VJp6DO1an6-p/file Resolving uc274ec77fe3055a2ffd2a56bf7e.dl.dropboxusercontent.com (uc274ec77fe3055a2ffd2a56bf7e.dl.dropboxusercontent.com)... 162.125.5.15, 2620:100:601d:15::a27d:50f Connecting to uc274ec77fe3055a2ffd2a56bf7e.dl.dropboxusercontent.com (uc274ec77fe3055a2ffd2a56bf7e.dl.dropboxusercontent.com)|162.125.5.15|:443... connected. HTTP request sent, awaiting response... 302 Found Location: /cd/0/inline2/BMlLJacxJ0nwQV0YhbbMX6yEqPni-sOrtLgY_la83a7g56VuTX0u5_bhL90m11QJPdtHttI2CCWgliGsXAe4m04P7PzIDInRmm1np2eubrjsB4o7aofUKf10X2v8t64XTD-rkhQf79co7ik0rFtwYVqPn1j45AqyCFoM_cHwF6uGDrYwzs7iq54KTy2UpWqH2geO0adFMLQUk7N2mKo8yhyQdM6thywOoWcifOPn3-FI26uscjRYm3gct2XgirHnZmVEs-YM5T8YjitlMe3NnlHBCyaodxBEr0jzAeUmHGKmi9jFISi8ITCR_DnERNC78AHjASe52n27L0AqfIvlJgihFG_bXKvkC-y3uWLUoouns5SCjhBQGM4gb0qFGPSt4Lw/file [following] --2021-04-14 14:44:01-- https://uc274ec77fe3055a2ffd2a56bf7e.dl.dropboxusercontent.com/cd/0/inline2/BMlLJacxJ0nwQV0YhbbMX6yEqPni-sOrtLgY_la83a7g56VuTX0u5_bhL90m11QJPdtHttI2CCWgliGsXAe4m04P7PzIDInRmm1np2eubrjsB4o7aofUKf10X2v8t64XTD-rkhQf79co7ik0rFtwYVqPn1j45AqyCFoM_cHwF6uGDrYwzs7iq54KTy2UpWqH2geO0adFMLQUk7N2mKo8yhyQdM6thywOoWcifOPn3-FI26uscjRYm3gct2XgirHnZmVEs-YM5T8YjitlMe3NnlHBCyaodxBEr0jzAeUmHGKmi9jFISi8ITCR_DnERNC78AHjASe52n27L0AqfIvlJgihFG_bXKvkC-y3uWLUoouns5SCjhBQGM4gb0qFGPSt4Lw/file Reusing existing connection to uc274ec77fe3055a2ffd2a56bf7e.dl.dropboxusercontent.com:443. HTTP request sent, awaiting response... 200 OK Length: 8725 (8.5K) [image/x-corelphotopaint] Saving to: ‘IR4AVHRR6.cpt’ IR4AVHRR6.cpt 100%[===================>] 8.52K --.-KB/s in 0s 2021-04-14 14:44:01 (117 MB/s) - ‘IR4AVHRR6.cpt’ saved [8725/8725]
!wget -c https://www.dropbox.com/s/i8j1g9xg1g7o56l/utilities.py
--2021-04-14 14:44:01-- https://www.dropbox.com/s/i8j1g9xg1g7o56l/utilities.py Resolving www.dropbox.com (www.dropbox.com)... 162.125.5.18, 2620:100:601d:18::a27d:512 Connecting to www.dropbox.com (www.dropbox.com)|162.125.5.18|:443... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: /s/raw/i8j1g9xg1g7o56l/utilities.py [following] --2021-04-14 14:44:01-- https://www.dropbox.com/s/raw/i8j1g9xg1g7o56l/utilities.py Reusing existing connection to www.dropbox.com:443. HTTP request sent, awaiting response... 302 Found Location: https://ucddc164850e7b087ac12dbafeee.dl.dropboxusercontent.com/cd/0/inline/BMksLo1LjcU8g18ryp7wMAIRPHICWphqHF0WknYBNucdtegK5akprb0vs7u8Yg1rsLrpXq9lGtW7MOfCpd9sktkVj4S2kbvgj0jT7GdJnWsCUGAiVVpbfnDoIVCiYOCZRZjLii-Soz7WKChLsHeB7xjE/file# [following] --2021-04-14 14:44:02-- https://ucddc164850e7b087ac12dbafeee.dl.dropboxusercontent.com/cd/0/inline/BMksLo1LjcU8g18ryp7wMAIRPHICWphqHF0WknYBNucdtegK5akprb0vs7u8Yg1rsLrpXq9lGtW7MOfCpd9sktkVj4S2kbvgj0jT7GdJnWsCUGAiVVpbfnDoIVCiYOCZRZjLii-Soz7WKChLsHeB7xjE/file Resolving ucddc164850e7b087ac12dbafeee.dl.dropboxusercontent.com (ucddc164850e7b087ac12dbafeee.dl.dropboxusercontent.com)... 162.125.5.15, 2620:100:601d:15::a27d:50f Connecting to ucddc164850e7b087ac12dbafeee.dl.dropboxusercontent.com (ucddc164850e7b087ac12dbafeee.dl.dropboxusercontent.com)|162.125.5.15|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 12040 (12K) [text/plain] Saving to: ‘utilities.py’ utilities.py 100%[===================>] 11.76K --.-KB/s in 0s 2021-04-14 14:44:02 (225 MB/s) - ‘utilities.py’ saved [12040/12040]
!wget -c https://www.dropbox.com/s/1ii0x5x87322rl5/pluvio_inmet_20201217.txt
--2021-04-14 14:44:02-- https://www.dropbox.com/s/1ii0x5x87322rl5/pluvio_inmet_20201217.txt Resolving www.dropbox.com (www.dropbox.com)... 162.125.5.18, 2620:100:601d:18::a27d:512 Connecting to www.dropbox.com (www.dropbox.com)|162.125.5.18|:443... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: /s/raw/1ii0x5x87322rl5/pluvio_inmet_20201217.txt [following] --2021-04-14 14:44:03-- https://www.dropbox.com/s/raw/1ii0x5x87322rl5/pluvio_inmet_20201217.txt Reusing existing connection to www.dropbox.com:443. HTTP request sent, awaiting response... 302 Found Location: https://ucc01cf30ebe82f39fc582438f6f.dl.dropboxusercontent.com/cd/0/inline/BMmUruTZmpN8_7xEYeWf2RHSDA-1ibUlyqXUEDoFSKINBWfDJfgzcTiQjVqsUKw3vUIpXJDbiMLWteBJhIoMY0InyRbfJPCv_jB3szXZ6074vKInVshDCOrBg3Fm6N1p1hxLUbe6O_Yhp9cY7KU-PHyx/file# [following] --2021-04-14 14:44:03-- https://ucc01cf30ebe82f39fc582438f6f.dl.dropboxusercontent.com/cd/0/inline/BMmUruTZmpN8_7xEYeWf2RHSDA-1ibUlyqXUEDoFSKINBWfDJfgzcTiQjVqsUKw3vUIpXJDbiMLWteBJhIoMY0InyRbfJPCv_jB3szXZ6074vKInVshDCOrBg3Fm6N1p1hxLUbe6O_Yhp9cY7KU-PHyx/file Resolving ucc01cf30ebe82f39fc582438f6f.dl.dropboxusercontent.com (ucc01cf30ebe82f39fc582438f6f.dl.dropboxusercontent.com)... 162.125.5.15, 2620:100:601d:15::a27d:50f Connecting to ucc01cf30ebe82f39fc582438f6f.dl.dropboxusercontent.com (ucc01cf30ebe82f39fc582438f6f.dl.dropboxusercontent.com)|162.125.5.15|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 19034 (19K) [text/plain] Saving to: ‘pluvio_inmet_20201217.txt’ pluvio_inmet_202012 100%[===================>] 18.59K --.-KB/s in 0.02s 2021-04-14 14:44:03 (755 KB/s) - ‘pluvio_inmet_20201217.txt’ saved [19034/19034]
!mkdir -p Samples
!mkdir -p Output
!wget -c https://noaa-goes16.s3.amazonaws.com/ABI-L2-CMIPF/2019/198/12/OR_ABI-L2-CMIPF-M6C08_G16_s20191981200396_e20191981210104_c20191981210182.nc -P /content/Samples/
!wget -c https://noaa-goes16.s3.amazonaws.com/ABI-L2-CMIPF/2019/198/12/OR_ABI-L2-CMIPF-M6C10_G16_s20191981200396_e20191981210116_c20191981210188.nc -P /content/Samples/
!wget -c https://noaa-goes16.s3.amazonaws.com/ABI-L2-CMIPF/2019/198/12/OR_ABI-L2-CMIPF-M6C12_G16_s20191981200396_e20191981210111_c20191981210185.nc -P /content/Samples/
!wget -c https://noaa-goes16.s3.amazonaws.com/ABI-L2-CMIPF/2019/198/12/OR_ABI-L2-CMIPF-M6C13_G16_s20191981200396_e20191981210116_c20191981210189.nc -P /content/Samples/
!wget -c https://noaa-goes16.s3.amazonaws.com/GLM-L2-LCFA/2019/198/12/OR_GLM-L2-LCFA_G16_s20191981200000_e20191981200200_c20191981200224.nc -P /content/Samples/
--2021-04-14 14:44:04-- https://noaa-goes16.s3.amazonaws.com/ABI-L2-CMIPF/2019/198/12/OR_ABI-L2-CMIPF-M6C08_G16_s20191981200396_e20191981210104_c20191981210182.nc Resolving noaa-goes16.s3.amazonaws.com (noaa-goes16.s3.amazonaws.com)... 52.216.112.251 Connecting to noaa-goes16.s3.amazonaws.com (noaa-goes16.s3.amazonaws.com)|52.216.112.251|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 19116775 (18M) [binary/octet-stream] Saving to: ‘/content/Samples/OR_ABI-L2-CMIPF-M6C08_G16_s20191981200396_e20191981210104_c20191981210182.nc’ OR_ABI-L2-CMIPF-M6C 100%[===================>] 18.23M 71.8MB/s in 0.3s 2021-04-14 14:44:04 (71.8 MB/s) - ‘/content/Samples/OR_ABI-L2-CMIPF-M6C08_G16_s20191981200396_e20191981210104_c20191981210182.nc’ saved [19116775/19116775] --2021-04-14 14:44:04-- https://noaa-goes16.s3.amazonaws.com/ABI-L2-CMIPF/2019/198/12/OR_ABI-L2-CMIPF-M6C10_G16_s20191981200396_e20191981210116_c20191981210188.nc Resolving noaa-goes16.s3.amazonaws.com (noaa-goes16.s3.amazonaws.com)... 52.216.112.251 Connecting to noaa-goes16.s3.amazonaws.com (noaa-goes16.s3.amazonaws.com)|52.216.112.251|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 21356777 (20M) [binary/octet-stream] Saving to: ‘/content/Samples/OR_ABI-L2-CMIPF-M6C10_G16_s20191981200396_e20191981210116_c20191981210188.nc’ OR_ABI-L2-CMIPF-M6C 100%[===================>] 20.37M 36.3MB/s in 0.6s 2021-04-14 14:44:05 (36.3 MB/s) - ‘/content/Samples/OR_ABI-L2-CMIPF-M6C10_G16_s20191981200396_e20191981210116_c20191981210188.nc’ saved [21356777/21356777] --2021-04-14 14:44:05-- https://noaa-goes16.s3.amazonaws.com/ABI-L2-CMIPF/2019/198/12/OR_ABI-L2-CMIPF-M6C12_G16_s20191981200396_e20191981210111_c20191981210185.nc Resolving noaa-goes16.s3.amazonaws.com (noaa-goes16.s3.amazonaws.com)... 52.217.15.44 Connecting to noaa-goes16.s3.amazonaws.com (noaa-goes16.s3.amazonaws.com)|52.217.15.44|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 23875622 (23M) [binary/octet-stream] Saving to: ‘/content/Samples/OR_ABI-L2-CMIPF-M6C12_G16_s20191981200396_e20191981210111_c20191981210185.nc’ OR_ABI-L2-CMIPF-M6C 100%[===================>] 22.77M 40.0MB/s in 0.6s 2021-04-14 14:44:06 (40.0 MB/s) - ‘/content/Samples/OR_ABI-L2-CMIPF-M6C12_G16_s20191981200396_e20191981210111_c20191981210185.nc’ saved [23875622/23875622] --2021-04-14 14:44:06-- https://noaa-goes16.s3.amazonaws.com/ABI-L2-CMIPF/2019/198/12/OR_ABI-L2-CMIPF-M6C13_G16_s20191981200396_e20191981210116_c20191981210189.nc Resolving noaa-goes16.s3.amazonaws.com (noaa-goes16.s3.amazonaws.com)... 52.216.153.196 Connecting to noaa-goes16.s3.amazonaws.com (noaa-goes16.s3.amazonaws.com)|52.216.153.196|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 26480642 (25M) [binary/octet-stream] Saving to: ‘/content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20191981200396_e20191981210116_c20191981210189.nc’ OR_ABI-L2-CMIPF-M6C 100%[===================>] 25.25M 50.2MB/s in 0.5s 2021-04-14 14:44:07 (50.2 MB/s) - ‘/content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20191981200396_e20191981210116_c20191981210189.nc’ saved [26480642/26480642] --2021-04-14 14:44:07-- https://noaa-goes16.s3.amazonaws.com/GLM-L2-LCFA/2019/198/12/OR_GLM-L2-LCFA_G16_s20191981200000_e20191981200200_c20191981200224.nc Resolving noaa-goes16.s3.amazonaws.com (noaa-goes16.s3.amazonaws.com)... 52.216.153.196 Connecting to noaa-goes16.s3.amazonaws.com (noaa-goes16.s3.amazonaws.com)|52.216.153.196|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 286335 (280K) [binary/octet-stream] Saving to: ‘/content/Samples/OR_GLM-L2-LCFA_G16_s20191981200000_e20191981200200_c20191981200224.nc’ OR_GLM-L2-LCFA_G16_ 100%[===================>] 279.62K --.-KB/s in 0.06s 2021-04-14 14:44:07 (4.38 MB/s) - ‘/content/Samples/OR_GLM-L2-LCFA_G16_s20191981200000_e20191981200200_c20191981200224.nc’ saved [286335/286335]
# Training: Python and GOES-R Imagery: Script 1 - Basic Plot / Extracting Pixel Values
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
#-----------------------------------------------------------------------------------------------------------
# Open the GOES-R image
# Download files at this link: http://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/goes16_download.cgi
file = Dataset("Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20191981200396_e20191981210116_c20191981210189.nc")
# Get the pixel values
data = file.variables['CMI'][:]
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(7,7), dpi=150)
# Plot the image
plt.imshow(data, vmin=193, vmax=313, cmap='Greys')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig('Output/Image_01.png')
# Show the image
plt.show()
import numpy as np
print(np.shape(data))
print(data[2500:3000,2500:3000])
(5424, 5424) [[285.9633483886719 286.33209228515625 287.1924133300781 ... 271.33746337890625 271.1531066894531 270.7843933105469] [286.9466247558594 286.33209228515625 286.7008056640625 ... 271.39892578125 270.90728759765625 269.9240417480469] [289.52764892578125 287.5611572265625 287.1309814453125 ... 270.47711181640625 270.2927551269531 269.67822265625] ... [268.81787109375 268.57208251953125 267.8960876464844 ... 276.5610046386719 276.1308288574219 276.315185546875] [270.04693603515625 269.248046875 268.6335144042969 ... 275.94647216796875 275.51629638671875 276.315185546875] [272.0134582519531 270.96875 269.30950927734375 ... 275.1475830078125 274.9017639160156 273.48834228515625]]
plt.figure(figsize=(7,7), dpi=150)
plt.plot(data[2500,:])
[<matplotlib.lines.Line2D at 0x7f4f72a9e990>]
# Training: Python and GOES-R Imagery: Script 2 - Basic Operation / Colorbar / Title / Date
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
#-----------------------------------------------------------------------------------------------------------
# Open the GOES-R image
# Download files at this link: http://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/goes16_download.cgi
file = Dataset("/content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20191981200396_e20191981210116_c20191981210189.nc")
# Get the pixel values
data = file.variables['CMI'][:] - 273.15
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(7,7), dpi=150)
# Plot the image
plt.imshow(data, vmin=-80, vmax=40, cmap='jet')
# Add a colorbar
plt.colorbar(label='Brightness Temperature (°C)', extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract date
date = (datetime.strptime(file.time_coverage_start, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 Band 13 ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Full Disk', fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig('/content/Output/Image_02.png')
# Show the image
plt.show()
# Training: Python and GOES-R Imagery: Script 3 - Adding a Map with Cartopy (Readin parameters from header)
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
#-----------------------------------------------------------------------------------------------------------
# Open the GOES-R image
# Download files at this link: http://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/goes16_download.cgi
file = Dataset("/content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20191981200396_e20191981210116_c20191981210189.nc")
# Get the pixel values
data = file.variables['CMI'][:] - 273.15
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(7,7), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.Geostationary(central_longitude=-75.0, satellite_height=35786023.0))
img_extent = (-5434894.67527,5434894.67527,-5434894.67527,5434894.67527)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='110m', color='white', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.5)
ax.gridlines(color='white', alpha=0.5, linestyle='--', linewidth=0.5)
# Plot the image
img = ax.imshow(data, vmin=-80, vmax=40, origin='upper', extent=img_extent, cmap='Greys')
# Add a colorbar
plt.colorbar(img, label='Brightness Temperatures (°C)', extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract date
date = (datetime.strptime(file.time_coverage_start, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 Band 13 ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Full Disk', fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig('/content/Output/Image_03.png')
# Show the image
plt.show()
/usr/local/lib/python3.7/dist-packages/cartopy/io/__init__.py:260: DownloadWarning: Downloading: https://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip
warnings.warn('Downloading: {}'.format(url), DownloadWarning)
/usr/local/lib/python3.7/dist-packages/cartopy/io/__init__.py:260: DownloadWarning: Downloading: https://naciscdn.org/naturalearth/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip
warnings.warn('Downloading: {}'.format(url), DownloadWarning)
file
<class 'netCDF4._netCDF4.Dataset'>
root group (NETCDF4 data model, file format HDF5):
naming_authority: gov.nesdis.noaa
Conventions: CF-1.7
Metadata_Conventions: Unidata Dataset Discovery v1.0
standard_name_vocabulary: CF Standard Name Table (v35, 20 July 2016)
institution: DOC/NOAA/NESDIS > U.S. Department of Commerce, National Oceanic and Atmospheric Administration, National Environmental Satellite, Data, and Information Services
project: GOES
production_site: NSOF
production_environment: OE
spatial_resolution: 2km at nadir
orbital_slot: GOES-East
platform_ID: G16
instrument_type: GOES R Series Advanced Baseline Imager
scene_id: Full Disk
instrument_ID: FM1
dataset_name: OR_ABI-L2-CMIPF-M6C13_G16_s20191981200396_e20191981210116_c20191981210189.nc
iso_series_metadata_id: 8c9e8150-3692-11e3-aa6e-0800200c9a66
title: ABI L2 Cloud and Moisture Imagery
summary: Single emissive band Cloud and Moisture Imagery Products are digital maps of clouds, moisture and atmospheric windows at IR bands.
keywords: SPECTRAL/ENGINEERING > INFRARED WAVELENGTHS > BRIGHTNESS TEMPERATURE
keywords_vocabulary: NASA Global Change Master Directory (GCMD) Earth Science Keywords, Version 7.0.0.0.0
license: Unclassified data. Access is restricted to approved users only.
processing_level: National Aeronautics and Space Administration (NASA) L2
date_created: 2019-07-17T12:10:18.9Z
cdm_data_type: Image
time_coverage_start: 2019-07-17T12:00:39.6Z
time_coverage_end: 2019-07-17T12:10:11.6Z
timeline_id: ABI Mode 6
production_data_source: n/a
id: 83b3d009-7840-4ea8-8531-052e0f68c4fc
dimensions(sizes): y(5424), x(5424), number_of_time_bounds(2), band(1), number_of_image_bounds(2)
variables(dimensions): int16 CMI(y, x), int8 DQF(y, x), float64 t(), int16 y(y), int16 x(x), float64 time_bounds(number_of_time_bounds), int32 goes_imager_projection(), float32 y_image(), float32 y_image_bounds(number_of_image_bounds), float32 x_image(), float32 x_image_bounds(number_of_image_bounds), float32 nominal_satellite_subpoint_lat(), float32 nominal_satellite_subpoint_lon(), float32 nominal_satellite_height(), float32 geospatial_lat_lon_extent(), float32 band_wavelength(band), int32 band_id(band), int32 total_number_of_points(), int32 valid_pixel_count(), int32 outlier_pixel_count(), float32 min_brightness_temperature(), float32 max_brightness_temperature(), float32 mean_brightness_temperature(), float32 std_dev_brightness_temperature(), float32 planck_fk1(), float32 planck_fk2(), float32 planck_bc1(), float32 planck_bc2(), int32 algorithm_dynamic_input_data_container(), float32 percent_uncorrectable_GRB_errors(), float32 percent_uncorrectable_L0_errors(), float32 earth_sun_distance_anomaly_in_AU(), int32 processing_parm_version_container(), int32 algorithm_product_version_container(), float32 esun(), float32 kappa0(), int32 focal_plane_temperature_threshold_exceeded_count(), float32 maximum_focal_plane_temperature(), float32 focal_plane_temperature_threshold_increasing(), float32 focal_plane_temperature_threshold_decreasing()
groups:
# Training: Python and GOES-R Imagery: Script 4 - Adding a Map with Cartopy
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
#-----------------------------------------------------------------------------------------------------------
# Open the GOES-R image
# Download files at this link: http://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/goes16_download.cgi
file = Dataset("/content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20191981200396_e20191981210116_c20191981210189.nc")
# Get the pixel values
data = file.variables['CMI'][:] - 273.15
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(7,7), dpi=150)
# Use the Geostationary projection in cartopy
longitude_of_projection_origin = file.variables['goes_imager_projection'].longitude_of_projection_origin
perspective_point_height = file.variables['goes_imager_projection'].perspective_point_height
ax = plt.axes(projection=ccrs.Geostationary(central_longitude=longitude_of_projection_origin, satellite_height=perspective_point_height))
# Extent of data in decimais (2712*0.000056*35786023.0)
xmin = file.variables['x'][:].min()*perspective_point_height
xmax = file.variables['x'][:].max()*perspective_point_height
ymin = file.variables['y'][:].min()*perspective_point_height
ymax = file.variables['y'][:].max()*perspective_point_height
img_extent = (xmin, xmax, ymin, ymax)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='110m', color='white', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.5)
ax.gridlines(color='white', alpha=0.5, linestyle='--', linewidth=0.5)
# Plot the image
img = ax.imshow(data, vmin=-80, vmax=40, origin='upper', extent=img_extent, cmap='Greys')
# Add a colorbar
plt.colorbar(img, label='Brightness Temperatures (°C)', extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract date
date = (datetime.strptime(file.time_coverage_start, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 Band 13 ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Full Disk', fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig('/content/Output/Image_04.png')
# Show the image
plt.show()
# Training: Python and GOES-R Imagery: Script 5 - Reading a Shapefile
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import cartopy.io.shapereader as shpreader # Import shapefiles
#-----------------------------------------------------------------------------------------------------------
# Open the GOES-R image
# Download files at this link: http://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/goes16_download.cgi
file = Dataset("/content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20191981200396_e20191981210116_c20191981210189.nc")
# Get the pixel values, and convert to Celsius
data = file.variables['CMI'][:] - 273.15
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(7,7), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.Geostationary(central_longitude=-75.0, satellite_height=35786023.0))
img_extent = (-5434894.67527,5434894.67527,-5434894.67527,5434894.67527)
# Add a shapefile
# https://geoftp.ibge.gov.br/organizacao_do_territorio/malhas_territoriais/malhas_municipais/municipio_2019/Brasil/BR/br_unidades_da_federacao.zip
# shapefile = list(shpreader.Reader('/content/BR_UF_2019.shp').geometries())
shapefile = list(shpreader.Reader('/content/ne_10m_admin_1_states_provinces.shp').geometries())
ax.add_geometries(shapefile, ccrs.PlateCarree(), edgecolor='gold',facecolor='none', linewidth=0.3)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='110m', color='white', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.8)
ax.gridlines(color='white', alpha=0.5, linestyle='--', linewidth=0.5)
# Plot the image
img = ax.imshow(data, vmin=-80, vmax=40, origin='upper', extent=img_extent, cmap='Greys')
# Add a colorbar
plt.colorbar(img, label='Brightness Temperature (°C)', extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract date
date = (datetime.strptime(file.time_coverage_start, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 Band 13 ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Full Disk', fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig('/content/Output/Image_05.png')
# Show the image
plt.show()
# Training: Python and GOES-R Imagery: Script 6 - ABI and GLM
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
#-----------------------------------------------------------------------------------------------------------
# Open the GOES-R image
# Download files at this link: http://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/goes16_download.cgi
file = Dataset("/content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20191981200396_e20191981210116_c20191981210189.nc")
# Get the pixel values
data = file.variables['CMI'][:] - 273.15
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(7,7), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.Geostationary(central_longitude=-75.0, satellite_height=35786023.0))
img_extent = (-5434894.67527,5434894.67527,-5434894.67527,5434894.67527)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='110m', color='white', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.5)
ax.gridlines(color='white', alpha=0.5, linestyle='--', linewidth=0.5)
# Plot the image
img = ax.imshow(data, vmin=-80, vmax=40, origin='upper', extent=img_extent, cmap='Greys')
#-----------------------------------------------------------------------------------------------------------
# Open the GLM file
# Download files at this link: http://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/goes16_download.cgi
fileGLM = Dataset("/content/Samples/OR_GLM-L2-LCFA_G16_s20191981200000_e20191981200200_c20191981200224.nc")
e_lats = fileGLM.variables['event_lat'][:]
e_lons = fileGLM.variables['event_lon'][:]
g_lats = fileGLM.variables['group_lat'][:]
g_lons = fileGLM.variables['group_lon'][:]
f_lats = fileGLM.variables['flash_lat'][:]
f_lons = fileGLM.variables['flash_lon'][:]
img2 = ax.plot(e_lons,e_lats,'.r', markersize=10, transform=ccrs.PlateCarree(), alpha=0.01)
img3 = ax.plot(g_lons,g_lats,'.y', markersize=5, transform=ccrs.PlateCarree(), alpha=0.5)
img4 = ax.plot(f_lons,f_lats,'.g', markersize=2.5, transform=ccrs.PlateCarree(), alpha=1)
#-----------------------------------------------------------------------------------------------------------
# Add a colorbar
plt.colorbar(img, label='Brightness Temperatures (°C)', extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract date
date = (datetime.strptime(file.time_coverage_start, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 Band 13 and GLM ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Full Disk', fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig('/content/Output/Image_06.png')
# Show the image
plt.show()
# Training: Python and GOES-R Imagery: Script 7 - RGB Creation
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import numpy as np # Import the Numpy packag
#-----------------------------------------------------------------------------------------------------------
# Open the GOES-R image
# Download files at this link: http://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/goes16_download.cgi
file1 = Dataset("/content/Samples/OR_ABI-L2-CMIPF-M6C08_G16_s20191981200396_e20191981210104_c20191981210182.nc")
# Get the pixel values, and convert to Celsius
data1 = file1.variables['CMI'][:,:][::4 ,::4] - 273.15
# Open the GOES-R image
file2 = Dataset("/content/Samples/OR_ABI-L2-CMIPF-M6C10_G16_s20191981200396_e20191981210116_c20191981210188.nc")
# Get the pixel values, and convert to Celsius
data2 = file2.variables['CMI'][:,:][::4 ,::4] - 273.15
# Open the GOES-R image
file3 = Dataset("/content/Samples/OR_ABI-L2-CMIPF-M6C12_G16_s20191981200396_e20191981210111_c20191981210185.nc")
# Get the pixel values, and convert to Celsius
data3 = file3.variables['CMI'][:,:][::4 ,::4] - 273.15
# Open the GOES-R image
file4 = Dataset("/content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20191981200396_e20191981210116_c20191981210189.nc")
# Get the pixel values, and convert to Celsius
data4 = file4.variables['CMI'][:,:][::4 ,::4] - 273.15
#-----------------------------------------------------------------------------------------------------------
# RGB Quick Guide: http://rammb.cira.colostate.edu/training/visit/quick_guides/QuickGuide_GOESR_AirMassRGB_final.pdf
# RGB Components
R = data1 - data2
G = data3 - data4
B = data1
# Minimuns, Maximuns and Gamma
Rmin = -26.2
Rmax = 0.6
Gmin = -43.2
Gmax = 6.7
Bmin = -29.25
Bmax = -64.65
R[R > Rmax] = Rmax
R[R < Rmin] = Rmin
G[G > Gmax] = Gmax
G[G < Gmin] = Gmin
B[B < Bmax] = Bmax # Inverted!
B[B > Bmin] = Bmin # Inverted!
gamma_R = 1
gamma_G = 1
gamma_B = 1
# Normalize the data
R = ((R - Rmin) / (Rmax - Rmin)) ** (1/gamma_R)
G = ((G - Gmin) / (Gmax - Gmin)) ** (1/gamma_G)
B = ((B - Bmin) / (Bmax - Bmin)) ** (1/gamma_B)
# Create the RGB
RGB = np.stack([R, G, B], axis=2)
# Eliminate values outside the globe
mask = (RGB == [R[0,0],G[0,0],B[0,0]]).all(axis=2)
RGB[mask] = np.nan
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(7,7), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.Geostationary(central_longitude=-75.0, satellite_height=35786023.0))
img_extent = (-5434894.67527,5434894.67527,-5434894.67527,5434894.67527)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='10m', color='white', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.8)
ax.gridlines(color='white', alpha=0.5, linestyle='--', linewidth=0.5)
# Plot the image
img = ax.imshow(RGB, origin='upper', extent=img_extent)
# Extract date
date = (datetime.strptime(file1.time_coverage_start, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 Airmass RGB ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Full Disk', fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig('/content/Output/Image_07.png')
# Show the image
plt.show()
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
/usr/local/lib/python3.7/dist-packages/cartopy/io/__init__.py:260: DownloadWarning: Downloading: https://naciscdn.org/naturalearth/10m/physical/ne_10m_coastline.zip
warnings.warn('Downloading: {}'.format(url), DownloadWarning)
# Training: Python and GOES-R Imagery: Script 8 - Custom Colormaps - Enhancing IR Channels
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
from matplotlib import cm # Colormap handling utilities
import numpy as np # Scientific computing with Python
from utilities import loadCPT # Import the CPT convert function
#-----------------------------------------------------------------------------------------------------------
# COLORMAP EXAMPLE 1
# Custom colormap joining matplotlib colormaps
# Available matplotlib colormaps: https://matplotlib.org/stable/tutorials/colors/colormaps.html
vmin1 = -80 # Min. value
vmax1 = 40 # Max. value
gray_cmap = cm.get_cmap('gray_r', 120) # Read the reversed 'gray' cmap
gray_cmap = gray_cmap(np.linspace(0, 1, 120)) # Create the array
jet_cmap = cm.get_cmap('jet_r', 40) # Read the reversed 'jet' cmap
jet_cmap = jet_cmap(np.linspace(0, 1, 40)) # Create the array
gray_cmap[:40, :] = jet_cmap # Join both cmaps arrays
my_cmap1 = cm.colors.ListedColormap(gray_cmap) # Create the custom colormap
#-----------------------------------------------------------------------------------------------------------
# COLORMAP EXAMPLE 2
# Creating the INPE DISSM IR colormap
# Online color picker: https://imagecolorpicker.com/
vmin2 = -80 # Min. value
vmax2 = 40 # Max. value
gray_cmap = cm.get_cmap('gray_r', 120) # Read the reversed 'gray' cmap
gray_cmap = gray_cmap(np.linspace(0, 1, 120)) # Create the array
colors = ["#ffa0ff", "#0806ff", "#3bcfff", "#feff65", "#ff7516"] # Custom colors
my_colors = cm.colors.ListedColormap(colors) # Create a custom colormap
my_colors = my_colors(np.linspace(0, 1, 50)) # Create the array
gray_cmap[:50, :] = my_colors # Join both cmaps arrays
my_cmap2 = cm.colors.ListedColormap(gray_cmap) # Create the custom colormap
#-----------------------------------------------------------------------------------------------------------
# COLORMAP EXAMPLE 3
# Creating a linear colormap
# Online color picker: https://imagecolorpicker.com/
colors = ["#bc8462", "#ae656f", "#a44a79", "#962e97", "#6158c5", "#2b8ffb", "#5fcdff", "#94fff0", "#a5ff94", "#fff88c", "#ffbf52", "#ec7b27", "#b84827", "#a1333d", "#bd5478", "#cc6a99", "#d982b8"]
my_cmap3 = cm.colors.LinearSegmentedColormap.from_list("", colors)# Create a custom linear colormap
vmin3 = -80 # Min. value
vmax3 = 40 # Max. value
#-----------------------------------------------------------------------------------------------------------
# COLORMAP EXAMPLE 4
# Converts a CPT file to be used in Python
# CPT archive: http://soliton.vm.bytemark.co.uk/pub/cpt-city/
cpt = loadCPT('/content/IR4AVHRR6.cpt') # Load the CPT file
my_cmap4 = cm.colors.LinearSegmentedColormap('cpt', cpt) # Create a custom linear colormap
vmin4 = -103.0 # Min. value
vmax4 = 84.0 # Max. value
#-----------------------------------------------------------------------------------------------------------
# Open the GOES-R image
# Download files at this link: http://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/goes16_download.cgi
file = Dataset("/content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20191981200396_e20191981210116_c20191981210189.nc")
# Get the pixel values
data = file.variables['CMI'][:] - 273.15
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
fig, axs = plt.subplots(2,2, figsize=(10,10), dpi=150) # 2 rows x 2 columns
#-----------------------------------------------------------------------------------------------------------
# Plot 1 (first row, first column)
# Plot the image
img1 = axs[0,0].imshow(data, vmin=vmin1, vmax=vmax1, origin='upper', cmap=my_cmap1)
# Add a colorbar
plt.colorbar(img1, extend='both', orientation='vertical', pad=0.05, fraction=0.05, ax=axs[0,0])
# Add a title
axs[0,0].set_title('Example 1 - Joining 2 cmaps', fontweight='bold', fontsize=10, loc='center')
#-----------------------------------------------------------------------------------------------------------
# Plot 2 (first row, second column)
# Plot the image
img2 = axs[0,1].imshow(data, vmin=vmin2, vmax=vmax2, origin='upper', cmap=my_cmap2)
# Add a colorbar
plt.colorbar(img2, label='Brightness Temperatures (°C)', extend='both', orientation='vertical', pad=0.05, fraction=0.05, ax=axs[0,1])
# Add a title
axs[0,1].set_title('Example 2 - Custom + cmap', fontweight='bold', fontsize=10, loc='center')
#-----------------------------------------------------------------------------------------------------------
# Plot 3 (second row, first column)
# Plot the image
img3 = axs[1,0].imshow(data, vmin=vmin3, vmax=vmax3, origin='upper', cmap=my_cmap3)
# Add a colorbar
plt.colorbar(img3, extend='both', orientation='vertical', pad=0.05, fraction=0.05, ax=axs[1,0])
# Add a title
axs[1,0].set_title('Example 3 - Only custom colors', fontweight='bold', fontsize=10, loc='center')
#-----------------------------------------------------------------------------------------------------------
# Plot 4 (second row, second column)
# Plot the image
img4 = axs[1,1].imshow(data, vmin=vmin4, vmax=vmax4, origin='upper', cmap=my_cmap4)
# Add a colorbar
plt.colorbar(img4, label='Brightness Temperatures (°C)', extend='both', orientation='vertical', pad=0.05, fraction=0.05, ax=axs[1,1])
# Add a title
axs[1,1].set_title('Example 4 - CPT file', fontweight='bold', fontsize=10, loc='center')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig('/content/Output/Image_08.png')
# Show the image
plt.show()
# Training: Python and GOES-R Imagery: Script 9 - Downloading data from AWS
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import os # Miscellaneous operating system interfaces
import boto3 # Amazon Web Services (AWS) SDK for Python
from botocore import UNSIGNED # boto3 config
from botocore.config import Config # boto3 config
#-----------------------------------------------------------------------------------------------------------
# Input and output directories
input = "/content/Samples"; os.makedirs(input, exist_ok=True)
output = "/content/Output"; os.makedirs(output, exist_ok=True)
# AMAZON repository information
# https://noaa-goes16.s3.amazonaws.com/index.html
bucket_name = 'noaa-goes16'
product_name = 'ABI-L2-CMIPF'
year = 2021
day_of_year = 91
hour = 18
min = 00
band = 2
# Initializes the S3 client
s3_client = boto3.client('s3', config=Config(signature_version=UNSIGNED))
#-----------------------------------------------------------------------------------------------------------
# File structure
prefix = f'{product_name}/{year}/{day_of_year:03.0f}/{hour:02.0f}/OR_{product_name}-M6C{band:02.0f}_G16_s{year}{day_of_year:03.0f}{hour:02.0f}{min:02.0f}'
# Seach for the file on the server
s3_result = s3_client.list_objects_v2(Bucket=bucket_name, Prefix=prefix, Delimiter = "/")
#-----------------------------------------------------------------------------------------------------------
# Check if there are files available
if 'Contents' not in s3_result:
# There are no files
print("No files found for the date: ",year,day_of_year)
quit()
else:
# There are files
for obj in s3_result['Contents']:
key = obj['Key']
# Print the file name
print(key)
file_name = key.split('/')[-1].split('.')[0]
# Download the file
if not os.path.exists(f'{input}/{file_name}.nc'):
s3_client.download_file(bucket_name, key, f'{input}/{file_name}.nc')
# If the file exists
if (os.path.exists(f'{input}/{file_name}.nc')):
# Open the GOES-R image
file = Dataset(f'{input}/{file_name}.nc')
# Get the pixel values
data = file.variables['CMI'][:]
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(10,10), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.Geostationary(central_longitude=-75.0, satellite_height=35786023.0))
img_extent = (-5434894.67527,5434894.67527,-5434894.67527,5434894.67527)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='10m', color='white', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.5)
ax.gridlines(color='white', alpha=0.5, linestyle='--', linewidth=0.5)
# Define the color scale based on the channel
if band <= 6:
colormap = "gray" # Black to white for visible channels
prodname = "Reflectance (%)"
else:
colormap = "gray_r" # White to black for IR channels
prodname = "Brightness Temperatures (°C)"
# Plot the image
img = ax.imshow(data, origin='upper', extent=img_extent, cmap=colormap)
# Add a colorbar
plt.colorbar(img, label=prodname, extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract date
date = (datetime.strptime(file.time_coverage_start, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 Band-' + str(band) + ' ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Full Disk', fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig(f'{output}/Image_09.png')
# Show the image
plt.show()
ABI-L2-CMIPF/2021/091/18/OR_ABI-L2-CMIPF-M6C02_G16_s20210911800179_e20210911809487_c20210911809552.nc
# Training: Python and GOES-R Imagery: Script 10 - Downloading data from AWS (function)
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import os # Miscellaneous operating system interfaces
import boto3 # Amazon Web Services (AWS) SDK for Python
from botocore import UNSIGNED # boto3 config
from botocore.config import Config # boto3 config
#-----------------------------------------------------------------------------------------------------------
# Function to download files
def download_file(s3_client, prefix):
# Seach for the file on the server
s3_result = s3_client.list_objects_v2(Bucket=bucket_name, Prefix=prefix, Delimiter = "/")
# Check if there are files available
if 'Contents' not in s3_result:
# There are no files
print("No files found for the date: ",year,day_of_year)
quit()
else:
# There are files
for obj in s3_result['Contents']:
# Print the file name
key = obj['Key']
print(key)
file_name = key.split('/')[-1].split('.')[0]
# Download the file
if not os.path.exists(f'{input}/{file_name}.nc'):
s3_client.download_file(bucket_name, key, f'{input}/{file_name}.nc')
return file_name
#-----------------------------------------------------------------------------------------------------------
# Input and output directories
input = "/content/Samples"; os.makedirs(input, exist_ok=True)
output = "/content/Output"; os.makedirs(output, exist_ok=True)
# AMAZON repository information
# https://noaa-goes16.s3.amazonaws.com/index.html
bucket_name = 'noaa-goes16'
product_name = 'ABI-L2-CMIPF'
year = 2021
day_of_year = 37
hour = 18
min = 00
band = 13
# Initializes the S3 client
s3_client = boto3.client('s3', config=Config(signature_version=UNSIGNED))
#-----------------------------------------------------------------------------------------------------------
# File structure
prefix = f'{product_name}/{year}/{day_of_year:03.0f}/{hour:02.0f}/OR_{product_name}-M6C{band:02.0f}_G16_s{year}{day_of_year:03.0f}{hour:02.0f}{min:02.0f}'
# Download the file
file_name = download_file(s3_client, prefix)
#-----------------------------------------------------------------------------------------------------------
# Open the GOES-R image
file = Dataset(f'{input}/{file_name}.nc')
# Get the pixel values
data = file.variables['CMI'][:]
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(10,10), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.Geostationary(central_longitude=-75.0, satellite_height=35786023.0))
img_extent = (-5434894.67527,5434894.67527,-5434894.67527,5434894.67527)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='10m', color='white', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.5)
ax.gridlines(color='white', alpha=0.5, linestyle='--', linewidth=0.5)
# Define the color scale based on the channel
if band <= 6:
colormap = "gray" # Black to white for visible channels
prodname = "Reflectance (%)"
else:
colormap = "gray_r" # White to black for IR channels
prodname = "Brightness Temperatures (°C)"
# Plot the image
img = ax.imshow(data, origin='upper', extent=img_extent, cmap=colormap)
# Add a colorbar
plt.colorbar(img, label=prodname, extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract date
date = (datetime.strptime(file.time_coverage_start, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 Band-' + str(band) + ' ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Full Disk', fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig(f'{output}/Image_10.png')
# Show the image
plt.show()
ABI-L2-CMIPF/2021/037/18/OR_ABI-L2-CMIPF-M6C13_G16_s20210371800122_e20210371809441_c20210371809524.nc
# Training: Python and GOES-R Imagery: Script 11 - Downloading data from AWS import function
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import os # Miscellaneous operating system interfaces
from utilities import download_CMI # INPE's utilities
#-----------------------------------------------------------------------------------------------------------
# Input and output directories
input = "/content/Samples"; os.makedirs(input, exist_ok=True)
output = "/content/Output"; os.makedirs(output, exist_ok=True)
# Datetime to process
yyyymmddhhmn = '202102181800'
band = '9'
# Download the file
file_name = download_CMI(yyyymmddhhmn, band, input)
#-----------------------------------------------------------------------------------------------------------
# Open the GOES-R image
file = Dataset(f'{input}/{file_name}.nc')
# Get the pixel values
data = file.variables['CMI'][:]
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(10,10), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.Geostationary(central_longitude=-75.0, satellite_height=35786023.0))
img_extent = (-5434894.67527,5434894.67527,-5434894.67527,5434894.67527)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='10m', color='white', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.5)
ax.gridlines(color='white', alpha=0.5, linestyle='--', linewidth=0.5)
# Define the color scale based on the channel
colormap = "gray_r" # White to black for IR channels
# Plot the image
img = ax.imshow(data, origin='upper', extent=img_extent, cmap=colormap)
# Add a colorbar
plt.colorbar(img, label='Brightness Temperatures (°C)', extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract date
date = (datetime.strptime(file.time_coverage_start, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 Band 9 ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Full Disk', fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig(f'{output}/Image_11.png')
# Show the image
plt.show()
Downloading file /content/Samples/OR_ABI-L2-CMIPF-M6C09_G16_s20210491800058_e20210491809372_c20210491809471.nc
# Training: Python and GOES-R Imagery: Script 12 - Cropping the Full Disk
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import os # Miscellaneous operating system interfaces
from utilities import download_CMI # Our own utilities
from utilities import geo2grid, latlon2xy, convertExtent2GOESProjection # Our own utilities
#-----------------------------------------------------------------------------------------------------------
# Input and output directories
input = "/content/Samples"; os.makedirs(input, exist_ok=True)
output = "/content/Output"; os.makedirs(output, exist_ok=True)
# Desired extent
extent = [-82.0, -5, -65.0, 16.0] # Min lon, Max lon, Min lat, Max lat
# Datetime to process
yyyymmddhhmn = '202103181800'
band = '13'
# Download the file
file_name = download_CMI(yyyymmddhhmn, band, input)
#-----------------------------------------------------------------------------------------------------------
# Open the GOES-R image
file = Dataset(f'{input}/{file_name}.nc')
# Convert lat/lon to grid-coordinates
lly, llx = geo2grid(extent[1], extent[0], file)
ury, urx = geo2grid(extent[3], extent[2], file)
# Get the pixel values
data = file.variables['CMI'][ury:lly, llx:urx]
#-----------------------------------------------------------------------------------------------------------
# Compute data-extent in GOES projection-coordinates
img_extent = convertExtent2GOESProjection(extent)
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(10,10), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.Geostationary(central_longitude=-75.0, satellite_height=35786023.0))
# Define the color scale based on the channel
colormap = "gray_r" # White to black for IR channels
# Plot the image
img = ax.imshow(data, origin='upper', extent=img_extent, cmap=colormap)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='10m', color='white', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.5)
ax.gridlines(color='white', alpha=0.5, linestyle='--', linewidth=0.5)
# Add a colorbar
plt.colorbar(img, label='Brightness Temperatures (°C)', extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract date
date = (datetime.strptime(file.time_coverage_start, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 Band 13 ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Reg.: ' + str(extent) , fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig(f'{output}/Image_12.png', bbox_inches='tight', pad_inches=0, dpi=300)
# Show the image
plt.show()
Downloading file /content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20210771800204_e20210771809523_c20210771809597.nc
/usr/local/lib/python3.7/dist-packages/cartopy/io/__init__.py:260: DownloadWarning: Downloading: https://naciscdn.org/naturalearth/50m/cultural/ne_50m_admin_0_boundary_lines_land.zip
warnings.warn('Downloading: {}'.format(url), DownloadWarning)
# Training: Python and GOES-R Imagery: Script 13 - Cropping the Full Disk and Creating an RGB
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import numpy as np # Scientific computing with Python
import os # Miscellaneous operating system interfaces
from utilities import download_CMI # Our own utilities
from utilities import geo2grid, latlon2xy, convertExtent2GOESProjection # Our own utilities
#-----------------------------------------------------------------------------------------------------------
# Input and output directories
input = "/content/Samples"; os.makedirs(input, exist_ok=True)
output = "/content/Output"; os.makedirs(output, exist_ok=True)
# Datetime to process
yyyymmddhhmn = '202102181800'
# Desired extent
extent = [-82.0, -5, -65.0, 16.0] # Min lon, Max lon, Min lat, Max lat
#-----------------------------------------------------------------------------------------------------------
# Download the necessary bands from AWS
file_ch13 = download_CMI(yyyymmddhhmn, 13, input)
file_ch02 = download_CMI(yyyymmddhhmn, 2, input)
file_ch05 = download_CMI(yyyymmddhhmn, 5, input)
#-----------------------------------------------------------------------------------------------------------
# Open the GOES-R images
file_ch13 = Dataset(f'{input}/{file_ch13}.nc')
file_ch02 = Dataset(f'{input}/{file_ch02}.nc')
file_ch05 = Dataset(f'{input}/{file_ch05}.nc')
#-----------------------------------------------------------------------------------------------------------
# Convert lat/lon to grid-coordinates
lly, llx = geo2grid(extent[1], extent[0], file_ch13)
ury, urx = geo2grid(extent[3], extent[2], file_ch13)
# Get the pixel values
data_ch13 = file_ch13.variables['CMI'][ury:lly, llx:urx] - 273.15
#-----------------------------------------------------------------------------------------------------------
# Convert lat/lon to grid-coordinates
lly, llx = geo2grid(extent[1], extent[0], file_ch02)
ury, urx = geo2grid(extent[3], extent[2], file_ch02)
# Get the pixel values
data_ch02 = file_ch02.variables['CMI'][ury:lly, llx:urx][::4 ,::4]
#-----------------------------------------------------------------------------------------------------------
# Convert lat/lon to grid-coordinates
lly, llx = geo2grid(extent[1], extent[0], file_ch05)
ury, urx = geo2grid(extent[3], extent[2], file_ch05)
# Get the pixel values
data_ch05 = file_ch05.variables['CMI'][ury:lly, llx:urx][::2 ,::2]
#-----------------------------------------------------------------------------------------------------------
print(np.shape(data_ch13),np.shape(data_ch05),np.shape(data_ch02))
# Make the arrays equal size
cordX = np.shape(data_ch02)[0], np.shape(data_ch05)[0], np.shape(data_ch13)[0]
cordY = np.shape(data_ch02)[1], np.shape(data_ch05)[1], np.shape(data_ch13)[1]
minvalX = np.array(cordX).min()
minvalY = np.array(cordY).min()
data_ch02 = data_ch02[0:minvalX, 0:minvalY]
data_ch05 = data_ch05[0:minvalX, 0:minvalY]
data_ch13 = data_ch13[0:minvalX, 0:minvalY]
#-----------------------------------------------------------------------------------------------------------
# Compute data-extent in GOES projection-coordinates
img_extent = convertExtent2GOESProjection(extent)
#-----------------------------------------------------------------------------------------------------------
# RGB Quick Guide: http://rammb.cira.colostate.edu/training/visit/quick_guides/QuickGuide_DayCloudConvectionRGB_final.pdf
# RGB Components
R = data_ch13
G = data_ch02
B = data_ch05
# Minimuns and Maximuns
Rmin = -53.5
Rmax = 7.5
Gmin = 0.0
Gmax = 0.78
Bmin = 0.01
Bmax = 0.59
R[R<Rmin] = Rmin
R[R>Rmax] = Rmax
G[G<Gmin] = Gmin
G[G>Gmax] = Gmax
B[B<Bmin] = Bmin
B[B>Bmax] = Bmax
# Choose the gamma
gamma = 1
# Normalize the data
R = ((R - Rmax) / (Rmin - Rmax)) ** (1/gamma)
G = ((G - Gmin) / (Gmax - Gmin)) ** (1/gamma)
B = ((B - Bmin) / (Bmax - Bmin)) ** (1/gamma)
print(np.shape(R),np.shape(G),np.shape(B))
# Create the RGB
RGB = np.stack([R, G, B], axis=2)
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(10,10), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.Geostationary(central_longitude=-75.0, satellite_height=35786023.0))
# Plot the image
img = ax.imshow(RGB, origin='upper', extent=img_extent)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='10m', color='white', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.5)
ax.gridlines(color='white', alpha=0.5, linestyle='--', linewidth=0.5)
# Extract date
date = (datetime.strptime(file_ch13.time_coverage_start, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 DCP RGB ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Reg.: ' + str(extent) , fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig(f'{output}/Image_13.png', bbox_inches='tight', pad_inches=0, dpi=300)
# Show the image
plt.show()
Downloading file /content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20210491800058_e20210491809378_c20210491809458.nc Downloading file /content/Samples/OR_ABI-L2-CMIPF-M6C02_G16_s20210491800058_e20210491809366_c20210491809450.nc Downloading file /content/Samples/OR_ABI-L2-CMIPF-M6C05_G16_s20210491800058_e20210491809366_c20210491809454.nc (1138, 912) (1138, 911) (1138, 912) (1138, 911) (1138, 911) (1138, 911)
# Training: Python and GOES-R Imagery: Script 14 - Reprojection with GDAL
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import os # Miscellaneous operating system interfaces
from osgeo import osr # Python bindings for GDAL
from osgeo import gdal # Python bindings for GDAL
import numpy as np # Scientific computing with Python
from utilities import download_CMI # Our function for download
#-----------------------------------------------------------------------------------------------------------
# Input and output directories
input = "/content/Samples"; os.makedirs(input, exist_ok=True)
output = "/content/Output"; os.makedirs(output, exist_ok=True)
# Desired extent
extent = [-82.0, -5, -65.0, 16.0] # Min lon, Max lon, Min lat, Max lat
# Datetime to process
yyyymmddhhmn = '202103181800'
band = '13'
# Download the file
file_name = download_CMI(yyyymmddhhmn, band, input)
#-----------------------------------------------------------------------------------------------------------
# Variable
var = 'CMI'
# Open the file
img = gdal.Open(f'NETCDF:{input}/{file_name}.nc:' + var)
# Read the header metadata
metadata = img.GetMetadata()
scale = float(metadata.get(var + '#scale_factor'))
offset = float(metadata.get(var + '#add_offset'))
undef = float(metadata.get(var + '#_FillValue'))
dtime = metadata.get('NC_GLOBAL#time_coverage_start')
# Load the data
ds = img.ReadAsArray(0, 0, img.RasterXSize, img.RasterYSize).astype(float)
# Apply the scale, offset and convert to celsius
ds = (ds * scale + offset) - 273.15
# Read the original file projection and configure the output projection
source_prj = osr.SpatialReference()
source_prj.ImportFromProj4(img.GetProjectionRef())
target_prj = osr.SpatialReference()
target_prj.ImportFromProj4("+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs")
# Reproject the data
GeoT = img.GetGeoTransform()
driver = gdal.GetDriverByName('MEM')
raw = driver.Create('raw', ds.shape[0], ds.shape[1], 1, gdal.GDT_Float32)
raw.SetGeoTransform(GeoT)
raw.GetRasterBand(1).WriteArray(ds)
# Define the parameters of the output file
options = gdal.WarpOptions(format = 'netCDF',
srcSRS = source_prj,
dstSRS = target_prj,
outputBounds = (extent[0], extent[3], extent[2], extent[1]),
outputBoundsSRS = target_prj,
outputType = gdal.GDT_Float32,
srcNodata = undef,
dstNodata = 'nan',
xRes = 0.02,
yRes = 0.02,
resampleAlg = gdal.GRA_NearestNeighbour)
print(options)
# Write the reprojected file on disk
gdal.Warp(f'{output}/{file_name}_ret.nc', raw, options=options)
#-----------------------------------------------------------------------------------------------------------
# Open the reprojected GOES-R image
file = Dataset(f'{output}/{file_name}_ret.nc')
# Get the pixel values
data = file.variables['Band1'][:]
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(10,10), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.PlateCarree())
# Define the image extent
img_extent = [extent[0], extent[2], extent[1], extent[3]]
# Define the color scale based on the channel
colormap = "gray_r" # White to black for IR channels
# Plot the image
img = ax.imshow(data, origin='upper', extent=img_extent, cmap=colormap)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='10m', color='white', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.5)
gl = ax.gridlines(crs=ccrs.PlateCarree(), color='gray', alpha=1.0, linestyle='--', linewidth=0.25, xlocs=np.arange(-180, 180, 5), ylocs=np.arange(-90, 90, 5), draw_labels=True)
gl.top_labels = False
gl.right_labels = False
# Add a colorbar
plt.colorbar(img, label='Brightness Temperatures (°C)', extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract date
date = (datetime.strptime(dtime, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 Band 13 ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Reg.: ' + str(extent) , fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig(f'{output}/Image_14.png', bbox_inches='tight', pad_inches=0, dpi=300)
# Show the image
plt.show()
File /content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20210771800204_e20210771809523_c20210771809597.nc exists (<osgeo.gdal.GDALWarpAppOptions; proxy of <Swig Object of type 'GDALWarpAppOptions *' at 0x7f4f4b39cb10> >, None, None)
# Primero se debe instalar esta librería para poder ejecutar el script siguiente.
pip install pyproj
Collecting pyproj
Downloading pyproj-3.0.1-cp37-cp37m-manylinux2010_x86_64.whl (6.5 MB)
|████████████████████████████████| 6.5 MB 4.7 MB/s
Requirement already satisfied: certifi in /usr/local/lib/python3.7/dist-packages (from pyproj) (2020.12.5)
Installing collected packages: pyproj
Successfully installed pyproj-3.0.1
# Training: Python and GOES-R Imagery: Script 14 - Reprojection with GDAL
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import cartopy.io.shapereader as shpreader # Import shapefiles
import os # Miscellaneous operating system interfaces
from osgeo import osr # Python bindings for GDAL
from osgeo import gdal # Python bindings for GDAL
import numpy as np # Scientific computing with Python
from utilities import download_CMI # Our function for download
gdal.PushErrorHandler('CPLQuietErrorHandler') # Ignore GDAL warnings
#-----------------------------------------------------------------------------------------------------------
# Input and output directories
input = "Samples"; os.makedirs(input, exist_ok=True)
output = "Output"; os.makedirs(output, exist_ok=True)
# Desired extent
extent = [-82.0, -5, -65.0, 16.0] # Min lon, Max lon, Min lat, Max lat
#------------------------------------------------
# Converting extents from epsg:4326 to epsg:3857
# conda install -c conda-forge pyproj
import pyproj
proj = pyproj.Transformer.from_crs(4326, 3857, always_xy=True)
x1,y1 = (extent[0], extent[1])
x2,y2 = (extent[2], extent[3])
a, b = proj.transform(x1, y1)
c, d = proj.transform(x2, y2)
extent_mercator = [a,b,c,d]
print(extent_mercator)
#------------------------------------------------
# AMAZON repository information
# https://noaa-goes16.s3.amazonaws.com/index.html
bucket_name = 'noaa-goes16'
product_name = 'ABI-L2-CMIPF'
yyyymmddhhmn = '202102181800'
band = '13'
# Download the file
file_name = download_CMI(yyyymmddhhmn, band, input)
#-----------------------------------------------------------------------------------------------------------
# Variable
var = 'CMI'
# Open the file
img = gdal.Open(f'NETCDF:{input}/{file_name}.nc:' + var)
# Read the header metadata
metadata = img.GetMetadata()
scale = float(metadata.get(var + '#scale_factor'))
offset = float(metadata.get(var + '#add_offset'))
undef = float(metadata.get(var + '#_FillValue'))
dtime = metadata.get('NC_GLOBAL#time_coverage_start')
# Load the data
ds = img.ReadAsArray(0, 0, img.RasterXSize, img.RasterYSize).astype(float)
# Apply the scale, offset
ds = (ds * scale + offset) - 273.15 # Convert to celsius
#-----------------------------------------------------------------------------------------------------------
# Read the original file projection and configure the output projection
source_prj = osr.SpatialReference()
source_prj.ImportFromProj4(img.GetProjectionRef())
target_prj = osr.SpatialReference()
target_prj.ImportFromProj4("+proj=merc +ellps=WGS84 +datum=WGS84 +units=m +no_defs")
# Reproject the data
GeoT = img.GetGeoTransform()
driver = gdal.GetDriverByName('netCDF')
raw = driver.Create('raw', ds.shape[0], ds.shape[1], 1, gdal.GDT_Float32)
raw.SetGeoTransform(GeoT)
raw.GetRasterBand(1).WriteArray(ds)
# Define the parameters of the output file
kwargs = {'format': 'netCDF', \
'srcSRS': source_prj, \
'dstSRS': target_prj, \
'outputBounds': (extent_mercator[0], extent_mercator[3], extent_mercator[2], extent_mercator[1]), \
'outputBoundsSRS': target_prj, \
'outputType': gdal.GDT_Float32, \
'srcNodata': undef, \
'dstNodata': 'nan', \
'xRes': 2000, \
'yRes': 2000, \
'resampleAlg': gdal.GRA_NearestNeighbour}
# Write the reprojected file on disk
gdal.Warp(f'{output}/{file_name}_ret.nc', raw, **kwargs)
print("Reprojection finished. Plotting the data...")
#-----------------------------------------------------------------------------------------------------------
# Open the reprojected GOES-R image
file = Dataset(f'{output}/{file_name}_ret.nc')
# Get the pixel values
data = file.variables['Band1'][:]
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(10,10), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.Mercator())
# Define the image extent
img_extent = [extent_mercator[0], extent_mercator[2], extent_mercator[1], extent_mercator[3]]
# Define the color scale based on the channel
colormap = "Greys"
# Plot the image
img = ax.imshow(data, vmin=-80, vmax=40, origin='upper', extent=img_extent, cmap=colormap)
# Add a shapefile
shapefile = list(shpreader.Reader('ne_10m_admin_1_states_provinces.shp').geometries())
ax.add_geometries(shapefile, ccrs.PlateCarree(), edgecolor='gold',facecolor='none', linewidth=0.3)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='10m', color='white', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.5)
gl = ax.gridlines(crs=ccrs.PlateCarree(), color='gray', alpha=1.0, linestyle='--', linewidth=0.25, xlocs=np.arange(-180, 180, 10), ylocs=np.arange(-90, 90, 10), draw_labels=True)
gl.top_labels = False
gl.right_labels = False
# Add a colorbar
plt.colorbar(img, label='Brightness Temperatures (°C)', extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract the date
date = (datetime.strptime(dtime, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 Band 13 ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
#plt.title('Reg.: ' + str(extent) , fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig(f'{output}/{file_name}_ret.png', bbox_inches='tight', pad_inches=0, dpi=300)
# Show the image
plt.show()
[-9128198.245048434, -557305.2572745769, -7235766.901562782, 1804722.766257292] File Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20210491800058_e20210491809378_c20210491809458.nc exists Reprojection finished. Plotting the data...
# Training: Python and GOES-R Imagery: Script 15 - Level 2 Products (SST) and Data Quality Flags
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import os # Miscellaneous operating system interfaces
from osgeo import gdal # Python bindings for GDAL
import numpy as np # Scientific computing with Python
from utilities import download_PROD # Our function for download
from utilities import reproject # Our function for reproject
gdal.PushErrorHandler('CPLQuietErrorHandler') # Ignore GDAL warnings
#-----------------------------------------------------------------------------------------------------------
#-----------------------------------------------------------------------------------------------------------
# Input and output directories
input = "/content/Samples"; os.makedirs(input, exist_ok=True)
output = "/content/Output"; os.makedirs(output, exist_ok=True)
# Desired extent
extent = [-82.0, -5, -65.0, 16.0] # Min lon, Max lon, Min lat, Max lat
# Parameters to process
yyyymmddhhmn = '202103281800'
product_name = 'ABI-L2-SSTF'
# Download the file
file_name = download_PROD(yyyymmddhhmn, product_name, input)
#-----------------------------------------------------------------------------------------------------------
# Variable
var = 'SST'
# Open the file
img = gdal.Open(f'NETCDF:{input}/{file_name}.nc:' + var)
# Data Quality Flag (DQF)
dqf = gdal.Open(f'NETCDF:{input}/{file_name}.nc:DQF')
# Read the header metadata
metadata = img.GetMetadata()
scale = float(metadata.get(var + '#scale_factor'))
offset = float(metadata.get(var + '#add_offset'))
undef = float(metadata.get(var + '#_FillValue'))
dtime = metadata.get('NC_GLOBAL#time_coverage_start')
# Load the data
ds = img.ReadAsArray(0, 0, img.RasterXSize, img.RasterYSize).astype(float)
ds_dqf = dqf.ReadAsArray(0, 0, dqf.RasterXSize, dqf.RasterYSize).astype(float)
# Apply the scale, offset and convert to celsius
ds = (ds * scale + offset) - 273.15
# Apply NaN's where the quality flag is greater than 1
ds[ds_dqf > 1] = np.nan
# Reproject the file
filename_ds = f'{output}/{file_name}_ret.nc'
reproject(filename_ds, img, ds, extent, undef)
#-----------------------------------------------------------------------------------------------------------
# Open the reprojected GOES-R image
file = Dataset(filename_ds)
# Get the pixel values
data = file.variables['Band1'][:]
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(10,10), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.PlateCarree())
# Define the image extent
img_extent = [extent[0], extent[2], extent[1], extent[3]]
# Plot the image
img = ax.imshow(data, vmin=8, vmax=35, cmap='jet', origin='upper', extent=img_extent)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='10m', color='black', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='black', linewidth=0.5)
gl = ax.gridlines(crs=ccrs.PlateCarree(), color='gray', alpha=1.0, linestyle='--', linewidth=0.25, xlocs=np.arange(-180, 180, 5), ylocs=np.arange(-90, 90, 5), draw_labels=True)
plt.xlim(extent[0], extent[2])
plt.ylim(extent[1], extent[3])
# Add a colorbar
plt.colorbar(img, label='Sea Surface Temperature (°C)', extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract date
date = (datetime.strptime(dtime, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 SST ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Reg.: ' + str(extent) , fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig(f'{output}/Image_15.png', bbox_inches='tight', pad_inches=0, dpi=300)
# Show the image
plt.show()
Downloading file /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210871800201_e20210871859509_c20210871905245.nc
# Training: Python and GOES-R Imagery: Script 16 - Level 2 Products (SST) and Average
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import os # Miscellaneous operating system interfaces
from osgeo import gdal # Python bindings for GDAL
import numpy as np # Scientific computing with Python
from utilities import download_PROD # Our function for download
from utilities import reproject # Our function for reproject
gdal.PushErrorHandler('CPLQuietErrorHandler') # Ignore GDAL warnings
#-----------------------------------------------------------------------------------------------------------
# Input and output directories
input = "/content/Samples"; os.makedirs(input, exist_ok=True)
output = "/content/Output"; os.makedirs(output, exist_ok=True)
# Parameters to process
yyyymmdd = '20210218'
product_name = 'ABI-L2-SSTF'
# Desired extent
extent = [-82.0, -5, -65.0, 16.0] # Min lon, Max lon, Min lat, Max lat
########################################################################
# Sea Surface Temperature - "X" Hours
########################################################################
sum_ds = np.zeros((5424,5424))
count_ds = np.zeros((5424,5424))
#-----------------------------------------------------------------------------------------------------------
for hour in np.arange(0,23,1):
# Date structure
yyyymmddhhmn = f'{yyyymmdd}{hour:02.0f}00'
# Download the file
file_name = download_PROD(yyyymmddhhmn, product_name, input)
#-----------------------------------------------------------------------------------------------------------
# Variable
var = 'SST'
# Open the GOES-R image
file = Dataset(f'{input}/{file_name}.nc')
# Open the file
img = gdal.Open(f'NETCDF:{input}/{file_name}.nc:' + var)
# Data Quality Flag (DQF)
dqf = gdal.Open(f'NETCDF:{input}/{file_name}.nc:DQF')
# Read the header metadata
metadata = img.GetMetadata()
scale = float(metadata.get(var + '#scale_factor'))
offset = float(metadata.get(var + '#add_offset'))
undef = float(metadata.get(var + '#_FillValue'))
dtime = metadata.get('NC_GLOBAL#time_coverage_start')
# Load the data
ds = img.ReadAsArray(0, 0, img.RasterXSize, img.RasterYSize).astype(float)
ds_dqf = dqf.ReadAsArray(0, 0, dqf.RasterXSize, dqf.RasterYSize).astype(float)
# Apply the scale, offset and convert to celsius
ds = (ds * scale + offset) - 273.15
# Apply NaN's where the quality flag is greater than 1
ds[ds_dqf > 1] = np.nan
# Calculate the sum
sum_ds = np.nansum(np.dstack((sum_ds,ds)),2)
count_ds = np.nansum(np.dstack((count_ds,(ds/ds))),2)
#-----------------------------------------------------------------------------------------------------------
# Calculate the sum
ds_day = np.empty((5424,5424))
ds_day[::] = np.nan
ds_day[count_ds!=0] = sum_ds[count_ds!=0]/count_ds[count_ds!=0]
#-----------------------------------------------------------------------------------------------------------
# Reproject the file
filename_ds = f'{output}/{file_name}_ret.nc'
reproject(filename_ds, img, ds_day, extent, undef)
#-----------------------------------------------------------------------------------------------------------
# Open the reprojected GOES-R image
file = Dataset(filename_ds)
# Get the pixel values
data = file.variables['Band1'][:]
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(10,10), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.PlateCarree())
# Define the image extent
img_extent = [extent[0], extent[2], extent[1], extent[3]]
# Plot the image
img = ax.imshow(data, vmin=15, vmax=30, cmap='jet', origin='upper', extent=img_extent)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='10m', color='black', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='black', linewidth=0.5)
gl = ax.gridlines(crs=ccrs.PlateCarree(), color='gray', alpha=1.0, linestyle='--', linewidth=0.25, xlocs=np.arange(-180, 180, 5), ylocs=np.arange(-90, 90, 5), draw_labels=True)
plt.xlim(extent[0], extent[2])
plt.ylim(extent[1], extent[3])
# Add a colorbar
plt.colorbar(img, label='Sea Surface Temperature (°C)', extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract date
date = (datetime.strptime(dtime, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('GOES-16 SST ' + date.strftime('%Y-%m-%d %H:%M') + ' UTC', fontweight='bold', fontsize=10, loc='left')
plt.title('Reg.: ' + str(extent) , fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig(f'{output}/Image_16.png', bbox_inches='tight', pad_inches=0, dpi=300)
# Show the image
plt.show()
File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210490000062_e20210490059370_c20210490105234.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210490100062_e20210490159369_c20210490205169.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210490200061_e20210490259369_c20210490305158.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210490300061_e20210490359369_c20210490405221.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210490400061_e20210490459369_c20210490505226.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210490500061_e20210490559371_c20210490605186.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210490600063_e20210490659371_c20210490705162.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210490700063_e20210490759370_c20210490805213.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210490800062_e20210490859370_c20210490905185.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210490900062_e20210490959370_c20210491005066.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210491000062_e20210491059370_c20210491105141.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210491100062_e20210491159369_c20210491205139.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210491200061_e20210491259369_c20210491305143.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210491300061_e20210491359369_c20210491405154.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210491400061_e20210491459369_c20210491505020.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210491500061_e20210491559369_c20210491605117.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210491600061_e20210491659368_c20210491705046.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210491700060_e20210491759366_c20210491805307.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210491800058_e20210491859366_c20210491905272.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210491900058_e20210491959366_c20210492005213.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210492000057_e20210492059365_c20210492105292.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210492100057_e20210492159365_c20210492205207.nc exists File /content/Samples/OR_ABI-L2-SSTF-M6_G16_s20210492200057_e20210492259365_c20210492305250.nc exists
# Training: Python and GOES-R Imagery: Script 17 - Level 2 Products (RRQPE) and Data Accumulation
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
from datetime import timedelta, date, datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import os # Miscellaneous operating system interfaces
from osgeo import gdal # Python bindings for GDAL
import numpy as np # Scientific computing with Python
from matplotlib import cm # Colormap handling utilities
from utilities import download_PROD # Our function for download
from utilities import reproject # Our function for reproject
gdal.PushErrorHandler('CPLQuietErrorHandler') # Ignore GDAL warnings
#-----------------------------------------------------------------------------------------------------------
# Input and output directories
input = "/content/Samples"; os.makedirs(input, exist_ok=True)
output = "/content/Output"; os.makedirs(output, exist_ok=True)
# Desired extent
extent = [-82.0, -5, -65.0, 16.0] # Min lon, Max lon, Min lat, Max lat
# Parameters to process
yyyymmdd = '20201217'
product_name = 'ABI-L2-RRQPEF'
########################################################################
# Rainfall Rate Quantitative Precipitation Estimation ("X" hours)
########################################################################
# Initial time and date
yyyy = datetime.strptime(yyyymmdd, '%Y%m%d').strftime('%Y')
mm = datetime.strptime(yyyymmdd, '%Y%m%d').strftime('%m')
dd = datetime.strptime(yyyymmdd, '%Y%m%d').strftime('%d')
date_ini = str(datetime(int(yyyy),int(mm),int(dd),12,0) - timedelta(hours=23))
date_end = str(datetime(int(yyyy),int(mm),int(dd),12,0))
acum = np.zeros((5424,5424))
#-----------------------------------------------------------------------------------------------------------
# Accumulation loop
while (date_ini <= date_end):
# Date structure
yyyymmddhhmn = datetime.strptime(date_ini, '%Y-%m-%d %H:%M:%S').strftime('%Y%m%d%H%M')
# Download the file
file_name = download_PROD(yyyymmddhhmn, product_name, input)
#-----------------------------------------------------------------------------------------------------------
# Variable
var = 'RRQPE'
# Open the file
img = gdal.Open(f'NETCDF:{input}/{file_name}.nc:' + var)
dqf = gdal.Open(f'NETCDF:{input}/{file_name}.nc:DQF')
# Read the header metadata
metadata = img.GetMetadata()
scale = float(metadata.get(var + '#scale_factor'))
offset = float(metadata.get(var + '#add_offset'))
undef = float(metadata.get(var + '#_FillValue'))
dtime = metadata.get('NC_GLOBAL#time_coverage_start')
# Load the data
ds = img.ReadAsArray(0, 0, img.RasterXSize, img.RasterYSize).astype(float)
ds_dqf = dqf.ReadAsArray(0, 0, dqf.RasterXSize, dqf.RasterYSize).astype(float)
# Remove undef
ds[ds == undef] = np.nan
# Apply the scale, offset and convert to celsius
ds = (ds * scale + offset)
# Apply NaN's where the quality flag is greater than 1
ds[ds_dqf > 0] = np.nan
# Sum the instantaneous value in the accumulation
acum = np.nansum(np.dstack((acum, ds)),2)
# Increment 1 hour
date_ini = str(datetime.strptime(date_ini, '%Y-%m-%d %H:%M:%S') + timedelta(hours=1))
#-----------------------------------------------------------------------------------------------------------
# Reproject the file
filename_acum = f'{output}/prec_acum_ret_{yyyymmddhhmn}.nc'
reproject(filename_acum, img, acum, extent, undef)
#-----------------------------------------------------------------------------------------------------------
# Open the reprojected GOES-R image
file = Dataset(filename_acum)
# Get the pixel values
data = file.variables['Band1'][:]
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(10,10), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.PlateCarree())
# Define the image extent
img_extent = [extent[0], extent[2], extent[1], extent[3]]
# Modify the colormap to zero values are white
colormap = cm.get_cmap('rainbow', 240)
newcolormap = colormap(np.linspace(0, 1, 240))
newcolormap[:1, :] = np.array([1, 1, 1, 1])
cmap = cm.colors.ListedColormap(newcolormap)
# Plot the image
img = ax.imshow(data, vmin=0, vmax=150, cmap=cmap, origin='upper', extent=img_extent)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='10m', color='black', linewidth=0.8)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='black', linewidth=0.5)
gl = ax.gridlines(crs=ccrs.PlateCarree(), color='gray', alpha=1.0, linestyle='--', linewidth=0.25, xlocs=np.arange(-180, 180, 5), ylocs=np.arange(-90, 90, 5), draw_labels=True)
plt.xlim(extent[0], extent[2])
plt.ylim(extent[1], extent[3])
# Add a colorbar
plt.colorbar(img, label='Rainfall Rate mm / 24h', extend='max', orientation='horizontal', pad=0.05, fraction=0.05)
# Extract date
date = (datetime.strptime(dtime, '%Y-%m-%dT%H:%M:%S.%fZ'))
# Add a title
plt.title('G-16 ACCUM. PREC.', fontweight='bold', fontsize=10, loc='left')
date_ini = datetime(int(yyyy),int(mm),int(dd),12,0) - timedelta(hours=23)
date_end = datetime(int(yyyy),int(mm),int(dd),12,0)
plt.title(date_ini.strftime('%Y-%m-%d %H:%M') + " - " + date_end.strftime('%Y-%m-%d %H:%M'), fontsize=10, loc='right')
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig(f'{output}/Image_17.png', bbox_inches='tight', pad_inches=0, dpi=300)
# Show the image
plt.show()
Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203511300221_e20203511309528_c20203511310017.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203511400220_e20203511409528_c20203511410016.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203511500220_e20203511509528_c20203511510017.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203511600220_e20203511609528_c20203511610019.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203511700218_e20203511709526_c20203511710014.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203511800218_e20203511809526_c20203511810055.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203511900218_e20203511909526_c20203511910055.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203512000217_e20203512009525_c20203512010050.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203512100217_e20203512109525_c20203512110016.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203512200217_e20203512209525_c20203512210022.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203512300217_e20203512309525_c20203512310024.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203520000217_e20203520009525_c20203520010025.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203520100217_e20203520109525_c20203520110016.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203520200217_e20203520209525_c20203520210010.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203520300216_e20203520309524_c20203520310015.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203520400216_e20203520409524_c20203520410015.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203520500218_e20203520509526_c20203520510017.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203520600218_e20203520609526_c20203520610018.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203520700218_e20203520709526_c20203520710010.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203520800218_e20203520809526_c20203520810025.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203520900218_e20203520909526_c20203520910020.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203521000218_e20203521009526_c20203521010040.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203521100218_e20203521109526_c20203521110027.nc Downloading file /content/Samples/OR_ABI-L2-RRQPEF-M6_G16_s20203521200218_e20203521209526_c20203521210044.nc
# Training: Python and GOES-R Imagery: Script 18 - Comparative of (RRQPE) with accumulated data of rain gauges
#-----------------------------------------------------------------------------------------------------------
import os
import matplotlib.pyplot as plt # Plotting library
import pandas as pd # Read and manipulate CSV file
from osgeo import gdal # Python bindings for GDAL
#-----------------------------------------------------------------------------------------------------------
# Input and output directories
input = "/content/Samples"; os.makedirs(input, exist_ok=True)
output = "/content/Output"; os.makedirs(output, exist_ok=True)
# Datetime to process
yyyymmdd = "20201217"
# Read file csv with observated rain
obs_data = pd.read_csv(f'pluvio_inmet_{yyyymmdd}.txt')
# Read file netcdf with estimated rain from GOES-16
sat_data = gdal.Open(f'{output}/prec_acum_ret_{yyyymmdd}1200.nc')
# Read number of cols and rows
ncol = sat_data.RasterXSize
nrow = sat_data.RasterYSize
# Load the data
sat_array = sat_data.ReadAsArray(0, 0, ncol, nrow).astype(float)
# Get geotransform
transform = sat_data.GetGeoTransform()
# Create a table with both of values
tab_prec = pd.DataFrame(columns=["lon","lat","obs","sat"])
idx = 0
for obs in obs_data["ACUM"]:
lon = obs_data.loc[idx,"LON"]
lat = obs_data.loc[idx,"LAT"]
x = int((lon - transform[0]) / transform[1])
y = int((transform[3] - lat) / -transform[5])
if x <= ncol and y <= nrow:
sat = sat_array[y,x]
tab_prec.loc[idx,"lon"] = lon
tab_prec.loc[idx,"lat"] = lat
tab_prec.loc[idx,"obs"] = obs
tab_prec.loc[idx,"sat"] = sat
idx += 1
fig, ax = plt.subplots(figsize=(10, 6))
ax.scatter(x = tab_prec['obs'], y = tab_prec['sat'])
plt.xlim(0, 100)
plt.ylim(0, 100)
ax.plot([0, 100], [0, 100], ls="--", c="r")
plt.xlabel("Observated (mm/24h)")
plt.ylabel("Estimated (mm/24h)")
plt.savefig(f'{output}/Image_18.png', bbox_inches='tight', pad_inches=0, dpi=300)
plt.show()
print(tab_prec)
Empty DataFrame Columns: [lon, lat, obs, sat] Index: []
# Training: Python and GOES-R Imagery: Script 19 - GLM Density
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
import cartopy, cartopy.crs as ccrs # Plot maps
from datetime import timedelta, date, datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import os # Miscellaneous operating system interfaces
from osgeo import gdal # Python bindings for GDAL
import numpy as np # Scientific computing with Python
from matplotlib import cm # Colormap handling utilities
from utilities import download_CMI, download_GLM # Our function for download
from utilities import reproject # Our function for reproject
gdal.PushErrorHandler('CPLQuietErrorHandler') # Ignore GDAL warnings
#-----------------------------------------------------------------------------------------------------------
# Input and output directories
input = "/content/Samples"; os.makedirs(input, exist_ok=True)
output = "/content/Output"; os.makedirs(output, exist_ok=True)
# Desired extent
extent = [-82.0, -5, -65.0, 16.0] # Min lon, Max lon, Min lat, Max lat
# Datetime to process
yyyymmddhhmn = '202102081800'
#-----------------------------------------------------------------------------------------------------------
# Get the Band 13 Data
# Download the file
file_ir = download_CMI(yyyymmddhhmn, 13, input)
#-----------------------------------------------------------------------------------------------------------
# Variable
var = 'CMI'
# Open the file
img = gdal.Open(f'NETCDF:{input}/{file_ir}.nc:' + var)
# Data Quality Flag (DQF)
dqf = gdal.Open(f'NETCDF:{input}/{file_ir}.nc:DQF')
# Read the header metadata
metadata = img.GetMetadata()
scale = float(metadata.get(var + '#scale_factor'))
offset = float(metadata.get(var + '#add_offset'))
undef = float(metadata.get(var + '#_FillValue'))
dtime = metadata.get('NC_GLOBAL#time_coverage_start')
# Load the data
ds_cmi = img.ReadAsArray(0, 0, img.RasterXSize, img.RasterYSize).astype(float)
ds_dqf = dqf.ReadAsArray(0, 0, dqf.RasterXSize, dqf.RasterYSize).astype(float)
# Apply the scale, offset and convert to celsius
ds_cmi = (ds_cmi * scale + offset) - 273.15
# Apply NaN's where the quality flag is greater than 1
ds_cmi[ds_dqf > 1] = np.nan
# Reproject the file
filename_ret = f'{output}/IR_{yyyymmddhhmn}.nc'
reproject(filename_ret, img, ds_cmi, extent, undef)
# Open the reprojected GOES-R image
file = Dataset(filename_ret)
# Get the pixel values
data = file.variables['Band1'][:]
#-----------------------------------------------------------------------------------------------------------
# Get the GLM Data
# Initialize arrays for latitude, longitude, and event energy
lats = np.array([])
lons = np.array([])
energies = np.array([])
#-----------------------------------------------------------------------------------------------------------
# Initial time and date
yyyy = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%Y')
mm = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%m')
dd = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%d')
hh = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%H')
mn = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%M')
date_ini = str(datetime(int(yyyy),int(mm),int(dd),int(hh),int(mn)))
date_end = str(datetime(int(yyyy),int(mm),int(dd),int(hh),int(mn)) + timedelta(minutes=10))
# GLM accumulation loop
while (date_ini <= date_end):
# Date structure
yyyymmddhhmnss = datetime.strptime(date_ini, '%Y-%m-%d %H:%M:%S').strftime('%Y%m%d%H%M%S')
# Download the file
file_glm = download_GLM(yyyymmddhhmnss, input)
# Read the file
glm = Dataset(f'{input}/{file_glm}.nc')
# Append lats / longs / event energies
lats = np.append(lats, glm.variables['event_lat'][:])
lons = np.append(lons, glm.variables['event_lon'][:])
energies = np.append(energies, glm.variables['event_energy'][:])
# Increment the date_ini
date_ini = str(datetime.strptime(date_ini, '%Y-%m-%d %H:%M:%S') + timedelta(seconds=20))
#-----------------------------------------------------------------------------------------------------------
# Stack and transpose the lat lons
values = np.vstack((lats, lons)).T
# Get the counts
points, counts = np.unique(values, axis=0, return_counts=True)
# Get the counts indices
idx = counts.argsort()
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(10,10), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.PlateCarree())
# Define the image extent
ax.set_extent([extent[0], extent[2], extent[1], extent[3]], ccrs.PlateCarree())
# Define the data extent
img_extent = [extent[0], extent[2], extent[1], extent[3]]
# Plot the image
img = ax.imshow(data, vmin=-80, vmax=40, cmap='gray_r', origin='upper', extent=img_extent, zorder=1)
# Plot the GLM Data
glm = plt.scatter(points[idx,1], points[idx,0], vmin=0, vmax=600, s=counts[idx]*0.1, c=counts[idx], cmap="jet", zorder=2)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='10m', color='white', linewidth=0.8, zorder=3)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.5, zorder=4)
gl = ax.gridlines(crs=ccrs.PlateCarree(), color='gray', alpha=1.0, linestyle='--', linewidth=0.25, xlocs=np.arange(-180, 180, 5), ylocs=np.arange(-90, 90, 5), draw_labels=True, zorder=5)
plt.xlim(extent[0], extent[2])
plt.ylim(extent[1], extent[3])
# Add the glm colorbar
plt.colorbar(glm, label='GLM Density', extend='max', orientation='vertical', pad=0.05, fraction=0.05)
# Add the img colorbar
plt.colorbar(img, label='Brightness Temperatures (°C)', extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Add a title
plt.title('GOES-16 IR+GLM', fontweight='bold', fontsize=10, loc='left')
date_ini = str(datetime(int(yyyy),int(mm),int(dd),int(hh),int(mn)))
date_end = str(datetime(int(yyyy),int(mm),int(dd),int(hh),int(mn)) + timedelta(minutes=10))
plt.title(str(date_ini) + " - " + str(date_end), fontsize="10", loc="right")
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig(f'{output}/Image_19.png', bbox_inches='tight', pad_inches=0, dpi=300)
# Show the image
plt.show()
File /content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20210391800111_e20210391809431_c20210391809523.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391800000_e20210391800204_c20210391800225.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391800200_e20210391800405_c20210391800420.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391800400_e20210391801004_c20210391801023.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391801000_e20210391801204_c20210391801227.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391801200_e20210391801404_c20210391801424.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391801400_e20210391802004_c20210391802022.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391802000_e20210391802204_c20210391802228.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391802200_e20210391802404_c20210391802427.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391802400_e20210391803004_c20210391803027.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391803000_e20210391803204_c20210391803228.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391803200_e20210391803404_c20210391803427.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391803400_e20210391804003_c20210391804026.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391804000_e20210391804204_c20210391804227.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391804200_e20210391804403_c20210391804429.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391804400_e20210391805004_c20210391805032.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391805000_e20210391805203_c20210391805226.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391805200_e20210391805404_c20210391805429.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391805400_e20210391806004_c20210391806026.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391806000_e20210391806204_c20210391806225.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391806200_e20210391806403_c20210391806426.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391806400_e20210391807003_c20210391807025.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391807000_e20210391807203_c20210391807220.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391807200_e20210391807403_c20210391807426.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391807400_e20210391808003_c20210391808025.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391808000_e20210391808205_c20210391808224.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391808200_e20210391808403_c20210391808428.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391808400_e20210391809003_c20210391809026.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391809000_e20210391809205_c20210391809223.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391809200_e20210391809405_c20210391809424.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391809400_e20210391810004_c20210391810026.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391810000_e20210391810204_c20210391810225.nc exists
# Training: Python and GOES-R Imagery: Script 20 - GLM Density
#-----------------------------------------------------------------------------------------------------------
# Required modules
from netCDF4 import Dataset # Read / Write NetCDF4 files
import matplotlib.pyplot as plt # Plotting library
import cartopy, cartopy.crs as ccrs # Plot maps
from datetime import timedelta, date, datetime # Basic Dates and time types
import cartopy, cartopy.crs as ccrs # Plot maps
import os # Miscellaneous operating system interfaces
from osgeo import gdal # Python bindings for GDAL
import numpy as np # Scientific computing with Python
from matplotlib import cm # Colormap handling utilities
from utilities import download_CMI, download_GLM # Our function for download
from utilities import reproject # Our function for reproject
from scipy.ndimage.filters import gaussian_filter # To make a heatmap
gdal.PushErrorHandler('CPLQuietErrorHandler') # Ignore GDAL warnings
#-----------------------------------------------------------------------------------------------------------
# Input and output directories
input = "/content/Samples"; os.makedirs(input, exist_ok=True)
output = "/content/Output"; os.makedirs(output, exist_ok=True)
# Desired extent
loni = -82
lonf = -5
lati = -65
latf = 16
extent = [loni, lonf, lati, latf] # Min lon, Max lon, Min lat, Max lat
# Datetime to process
yyyymmddhhmn = '202102081800'
#-----------------------------------------------------------------------------------------------------------
# Get the Band 13 Data
# Download the file
file_ir = download_CMI(yyyymmddhhmn, 13, input)
#-----------------------------------------------------------------------------------------------------------
# Variable
var = 'CMI'
# Open the file
img = gdal.Open(f'NETCDF:{input}/{file_ir}.nc:' + var)
# Data Quality Flag (DQF)
dqf = gdal.Open(f'NETCDF:{input}/{file_ir}.nc:DQF')
# Read the header metadata
metadata = img.GetMetadata()
scale = float(metadata.get(var + '#scale_factor'))
offset = float(metadata.get(var + '#add_offset'))
undef = float(metadata.get(var + '#_FillValue'))
dtime = metadata.get('NC_GLOBAL#time_coverage_start')
# Load the data
ds_cmi = img.ReadAsArray(0, 0, img.RasterXSize, img.RasterYSize).astype(float)
ds_dqf = dqf.ReadAsArray(0, 0, dqf.RasterXSize, dqf.RasterYSize).astype(float)
# Apply the scale, offset and convert to celsius
ds_cmi = (ds_cmi * scale + offset) - 273.15
# Apply NaN's where the quality flag is greater than 1
ds_cmi[ds_dqf > 1] = np.nan
# Reproject the file
filename_ret = f'{output}/IR_{yyyymmddhhmn}.nc'
reproject(filename_ret, img, ds_cmi, extent, undef)
# Open the reprojected GOES-R image
file = Dataset(filename_ret)
# Get the pixel values
data = file.variables['Band1'][:]
#-----------------------------------------------------------------------------------------------------------
# Get the GLM Data
# Initialize arrays for latitude, longitude, and event energy
lats = np.array([])
lons = np.array([])
#-----------------------------------------------------------------------------------------------------------
# Initial time and date
yyyy = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%Y')
mm = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%m')
dd = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%d')
hh = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%H')
mn = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%M')
date_ini = str(datetime(int(yyyy),int(mm),int(dd),int(hh),int(mn)))
date_end = str(datetime(int(yyyy),int(mm),int(dd),int(hh),int(mn)) + timedelta(minutes=10))
# GLM accumulation loop
while (date_ini <= date_end):
# Date structure
yyyymmddhhmnss = datetime.strptime(date_ini, '%Y-%m-%d %H:%M:%S').strftime('%Y%m%d%H%M%S')
# Download the file
file_glm = download_GLM(yyyymmddhhmnss, input)
# Read the file
glm = Dataset(f'{input}/{file_glm}.nc')
# Append lats / longs / event energies
lats = np.append(lats, glm.variables['event_lat'][:])
lons = np.append(lons, glm.variables['event_lon'][:])
# Increment the date_ini
date_ini = str(datetime.strptime(date_ini, '%Y-%m-%d %H:%M:%S') + timedelta(seconds=20))
#-----------------------------------------------------------------------------------------------------------
# Stack and transpose the lat lons
values = np.vstack((lons,lats)).T
values = values[(values[:,0] >= loni) & (values[:,0] <= lonf)]
values = values[(values[:,1] >= lati) & (values[:,1] <= latf)]
from scipy.ndimage.filters import gaussian_filter
heatmap, x, y = np.histogram2d(values[:,0], values[:,1], bins=(120,120), range=[[loni,lonf],[lati,latf]])
heatmap = gaussian_filter(heatmap,1)
print(heatmap.min(), heatmap.max(), heatmap.shape)
#-----------------------------------------------------------------------------------------------------------
# Choose the plot size (width x height, in inches)
plt.figure(figsize=(10,10), dpi=150)
# Use the Geostationary projection in cartopy
ax = plt.axes(projection=ccrs.PlateCarree())
# Define the image extent
ax.set_extent([extent[0], extent[2], extent[1], extent[3]], ccrs.PlateCarree())
# Define the data extent
img_extent = [extent[0], extent[2], extent[1], extent[3]]
# Plot the image
img = ax.imshow(data, vmin=-50, vmax=80, cmap='gray_r', origin='upper', extent=img_extent)
# Plot the GLM Data
glm = ax.imshow(heatmap.T, vmin=0, vmax=600, origin="lower", cmap="hot", interpolation="nearest", extent=img_extent, alpha=0.6)
# Add coastlines, borders and gridlines
ax.coastlines(resolution='10m', color='white', linewidth=0.8, zorder=3)
ax.add_feature(cartopy.feature.BORDERS, edgecolor='white', linewidth=0.5, zorder=4)
gl = ax.gridlines(crs=ccrs.PlateCarree(), color='gray', alpha=1.0, linestyle='--', linewidth=0.25, xlocs=np.arange(-180, 180, 5), ylocs=np.arange(-90, 90, 5), draw_labels=True, zorder=5)
plt.xlim(extent[0], extent[2])
plt.ylim(extent[1], extent[3])
# Add the img colorbar
#plt.colorbar(img, label='Brightness Temperatures (°C)', extend='both', orientation='horizontal', pad=0.05, fraction=0.05)
# Add the glm colorbar
plt.colorbar(glm, label='GLM Density', extend='max', orientation='vertical', pad=0.05, fraction=0.05)
# Add a title
plt.title('GOES-16 IR+GLM', fontweight='bold', fontsize=10, loc='left')
date_ini = str(datetime(int(yyyy),int(mm),int(dd),int(hh),int(mn)))
date_end = str(datetime(int(yyyy),int(mm),int(dd),int(hh),int(mn)) + timedelta(minutes=10))
plt.title(str(date_ini) + " - " + str(date_end), fontsize="10", loc="right")
#-----------------------------------------------------------------------------------------------------------
# Save the image
plt.savefig(f'{output}/Image_20.png', bbox_inches='tight', pad_inches=0, dpi=300)
# Show the image
plt.show()
File /content/Samples/OR_ABI-L2-CMIPF-M6C13_G16_s20210391800111_e20210391809431_c20210391809523.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391800000_e20210391800204_c20210391800225.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391800200_e20210391800405_c20210391800420.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391800400_e20210391801004_c20210391801023.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391801000_e20210391801204_c20210391801227.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391801200_e20210391801404_c20210391801424.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391801400_e20210391802004_c20210391802022.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391802000_e20210391802204_c20210391802228.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391802200_e20210391802404_c20210391802427.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391802400_e20210391803004_c20210391803027.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391803000_e20210391803204_c20210391803228.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391803200_e20210391803404_c20210391803427.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391803400_e20210391804003_c20210391804026.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391804000_e20210391804204_c20210391804227.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391804200_e20210391804403_c20210391804429.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391804400_e20210391805004_c20210391805032.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391805000_e20210391805203_c20210391805226.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391805200_e20210391805404_c20210391805429.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391805400_e20210391806004_c20210391806026.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391806000_e20210391806204_c20210391806225.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391806200_e20210391806403_c20210391806426.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391806400_e20210391807003_c20210391807025.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391807000_e20210391807203_c20210391807220.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391807200_e20210391807403_c20210391807426.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391807400_e20210391808003_c20210391808025.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391808000_e20210391808205_c20210391808224.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391808200_e20210391808403_c20210391808428.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391808400_e20210391809003_c20210391809026.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391809000_e20210391809205_c20210391809223.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391809200_e20210391809405_c20210391809424.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391809400_e20210391810004_c20210391810026.nc exists File /content/Samples/OR_GLM-L2-LCFA_G16_s20210391810000_e20210391810204_c20210391810225.nc exists 0.0 944.0201902011089 (120, 120)
# Apéndice del archivo utilities.py
!cat utilities.py
# Training: Python and GOES-R Imagery: Script 8 - Functions for download data from AWS
#-----------------------------------------------------------------------------------------------------------
# Required modules
import os # Miscellaneous operating system interfaces
import numpy as np # Import the Numpy package
import colorsys # To make convertion of colormaps
import boto3 # Amazon Web Services (AWS) SDK for Python
from botocore import UNSIGNED # boto3 config
from botocore.config import Config # boto3 config
import math # Mathematical functions
from datetime import datetime # Basic Dates and time types
from osgeo import osr # Python bindings for GDAL
from osgeo import gdal # Python bindings for GDAL
#from netCDF4 import Dataset # Read / Write NetCDF4 files
#import matplotlib.pyplot as plt # Plotting library
#import cartopy, cartopy.crs as ccrs # Plot maps
##import sys
#from datetime import timedelta, date, datetime # Manipulate dates
#-----------------------------------------------------------------------------------------------------------
def loadCPT(path):
try:
f = open(path)
except:
print ("File ", path, "not found")
return None
lines = f.readlines()
f.close()
x = np.array([])
r = np.array([])
g = np.array([])
b = np.array([])
colorModel = 'RGB'
for l in lines:
ls = l.split()
if l[0] == '#':
if ls[-1] == 'HSV':
colorModel = 'HSV'
continue
else:
continue
if ls[0] == 'B' or ls[0] == 'F' or ls[0] == 'N':
pass
else:
x=np.append(x,float(ls[0]))
r=np.append(r,float(ls[1]))
g=np.append(g,float(ls[2]))
b=np.append(b,float(ls[3]))
xtemp = float(ls[4])
rtemp = float(ls[5])
gtemp = float(ls[6])
btemp = float(ls[7])
x=np.append(x,xtemp)
r=np.append(r,rtemp)
g=np.append(g,gtemp)
b=np.append(b,btemp)
if colorModel == 'HSV':
for i in range(r.shape[0]):
rr, gg, bb = colorsys.hsv_to_rgb(r[i]/360.,g[i],b[i])
r[i] = rr ; g[i] = gg ; b[i] = bb
if colorModel == 'RGB':
r = r/255.0
g = g/255.0
b = b/255.0
xNorm = (x - x[0])/(x[-1] - x[0])
red = []
blue = []
green = []
for i in range(len(x)):
red.append([xNorm[i],r[i],r[i]])
green.append([xNorm[i],g[i],g[i]])
blue.append([xNorm[i],b[i],b[i]])
colorDict = {'red': red, 'green': green, 'blue': blue}
return colorDict
#-----------------------------------------------------------------------------------------------------------
def download_CMI(yyyymmddhhmn, band, path_dest):
os.makedirs(path_dest, exist_ok=True)
year = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%Y')
day_of_year = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%j')
hour = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%H')
min = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%M')
# AMAZON repository information
# https://noaa-goes16.s3.amazonaws.com/index.html
bucket_name = 'noaa-goes16'
product_name = 'ABI-L2-CMIPF'
# Initializes the S3 client
s3_client = boto3.client('s3', config=Config(signature_version=UNSIGNED))
#-----------------------------------------------------------------------------------------------------------
# File structure
prefix = f'{product_name}/{year}/{day_of_year}/{hour}/OR_{product_name}-M6C{int(band):02.0f}_G16_s{year}{day_of_year}{hour}{min}'
# Seach for the file on the server
s3_result = s3_client.list_objects_v2(Bucket=bucket_name, Prefix=prefix, Delimiter = "/")
#-----------------------------------------------------------------------------------------------------------
# Check if there are files available
if 'Contents' not in s3_result:
# There are no files
print(f'No files found for the date: {yyyymmddhhmn}, Band-{band}')
return -1
else:
# There are files
for obj in s3_result['Contents']:
key = obj['Key']
# Print the file name
file_name = key.split('/')[-1].split('.')[0]
# Download the file
if os.path.exists(f'{path_dest}/{file_name}.nc'):
print(f'File {path_dest}/{file_name}.nc exists')
else:
print(f'Downloading file {path_dest}/{file_name}.nc')
s3_client.download_file(bucket_name, key, f'{path_dest}/{file_name}.nc')
return f'{file_name}'
#-----------------------------------------------------------------------------------------------------------
def download_PROD(yyyymmddhhmn, product_name, path_dest):
os.makedirs(path_dest, exist_ok=True)
year = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%Y')
day_of_year = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%j')
hour = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%H')
min = datetime.strptime(yyyymmddhhmn, '%Y%m%d%H%M').strftime('%M')
# AMAZON repository information
# https://noaa-goes16.s3.amazonaws.com/index.html
bucket_name = 'noaa-goes16'
# Initializes the S3 client
s3_client = boto3.client('s3', config=Config(signature_version=UNSIGNED))
#-----------------------------------------------------------------------------------------------------------
# File structure
prefix = f'{product_name}/{year}/{day_of_year}/{hour}/OR_{product_name}-M6_G16_s{year}{day_of_year}{hour}{min}'
# Seach for the file on the server
s3_result = s3_client.list_objects_v2(Bucket=bucket_name, Prefix=prefix, Delimiter = "/")
#-----------------------------------------------------------------------------------------------------------
# Check if there are files available
if 'Contents' not in s3_result:
# There are no files
print(f'No files found for the date: {yyyymmddhhmn}, Product-{product_name}')
return -1
else:
# There are files
for obj in s3_result['Contents']:
key = obj['Key']
# Print the file name
file_name = key.split('/')[-1].split('.')[0]
# Download the file
if os.path.exists(f'{path_dest}/{file_name}.nc'):
print(f'File {path_dest}/{file_name}.nc exists')
else:
print(f'Downloading file {path_dest}/{file_name}.nc')
s3_client.download_file(bucket_name, key, f'{path_dest}/{file_name}.nc')
return f'{file_name}'
#-----------------------------------------------------------------------------------------------------------
def download_GLM(yyyymmddhhmnss, path_dest):
os.makedirs(path_dest, exist_ok=True)
year = datetime.strptime(yyyymmddhhmnss, '%Y%m%d%H%M%S').strftime('%Y')
day_of_year = datetime.strptime(yyyymmddhhmnss, '%Y%m%d%H%M%S').strftime('%j')
hour = datetime.strptime(yyyymmddhhmnss, '%Y%m%d%H%M%S').strftime('%H')
min = datetime.strptime(yyyymmddhhmnss, '%Y%m%d%H%M%S').strftime('%M')
seg = datetime.strptime(yyyymmddhhmnss, '%Y%m%d%H%M%S').strftime('%S')
# AMAZON repository information
# https://noaa-goes16.s3.amazonaws.com/index.html
bucket_name = 'noaa-goes16'
# Initializes the S3 client
s3_client = boto3.client('s3', config=Config(signature_version=UNSIGNED))
#-----------------------------------------------------------------------------------------------------------
# File structure
product_name = "GLM-L2-LCFA"
prefix = f'{product_name}/{year}/{day_of_year}/{hour}/OR_{product_name}_G16_s{year}{day_of_year}{hour}{min}{seg}'
# Seach for the file on the server
s3_result = s3_client.list_objects_v2(Bucket=bucket_name, Prefix=prefix, Delimiter = "/")
#-----------------------------------------------------------------------------------------------------------
# Check if there are files available
if 'Contents' not in s3_result:
# There are no files
print(f'No files found for the date: {yyyymmddhhmnss}, Product-{product_name}')
return -1
else:
# There are files
for obj in s3_result['Contents']:
key = obj['Key']
# Print the file name
file_name = key.split('/')[-1].split('.')[0]
# Download the file
if os.path.exists(f'{path_dest}/{file_name}.nc'):
print(f'File {path_dest}/{file_name}.nc exists')
else:
print(f'Downloading file {path_dest}/{file_name}.nc')
s3_client.download_file(bucket_name, key, f'{path_dest}/{file_name}.nc')
return f'{file_name}'
#-----------------------------------------------------------------------------------------------------------
# Functions to convert lat / lon extent to array indices
def geo2grid(lat, lon, nc):
# Apply scale and offset
xscale, xoffset = nc.variables['x'].scale_factor, nc.variables['x'].add_offset
yscale, yoffset = nc.variables['y'].scale_factor, nc.variables['y'].add_offset
x, y = latlon2xy(lat, lon)
col = (x - xoffset)/xscale
lin = (y - yoffset)/yscale
return int(lin), int(col)
def latlon2xy(lat, lon):
# goes_imagery_projection:semi_major_axis
req = 6378137 # meters
# goes_imagery_projection:inverse_flattening
invf = 298.257222096
# goes_imagery_projection:semi_minor_axis
rpol = 6356752.31414 # meters
e = 0.0818191910435
# goes_imagery_projection:perspective_point_height + goes_imagery_projection:semi_major_axis
H = 42164160 # meters
# goes_imagery_projection: longitude_of_projection_origin
lambda0 = -1.308996939
# Convert to radians
latRad = lat * (math.pi/180)
lonRad = lon * (math.pi/180)
# (1) geocentric latitude
Phi_c = math.atan(((rpol * rpol)/(req * req)) * math.tan(latRad))
# (2) geocentric distance to the point on the ellipsoid
rc = rpol/(math.sqrt(1 - ((e * e) * (math.cos(Phi_c) * math.cos(Phi_c)))))
# (3) sx
sx = H - (rc * math.cos(Phi_c) * math.cos(lonRad - lambda0))
# (4) sy
sy = -rc * math.cos(Phi_c) * math.sin(lonRad - lambda0)
# (5)
sz = rc * math.sin(Phi_c)
# x,y
x = math.asin((-sy)/math.sqrt((sx*sx) + (sy*sy) + (sz*sz)))
y = math.atan(sz/sx)
return x, y
# Function to convert lat / lon extent to GOES-16 extents
def convertExtent2GOESProjection(extent):
# GOES-16 viewing point (satellite position) height above the earth
GOES16_HEIGHT = 35786023.0
# GOES-16 longitude position
GOES16_LONGITUDE = -75.0
a, b = latlon2xy(extent[1], extent[0])
c, d = latlon2xy(extent[3], extent[2])
return (a * GOES16_HEIGHT, c * GOES16_HEIGHT, b * GOES16_HEIGHT, d * GOES16_HEIGHT)
#-----------------------------------------------------------------------------------------------------------
# Function to reproject the data
def reproject(file_name, ncfile, array, extent, undef):
# Read the original file projection and configure the output projection
source_prj = osr.SpatialReference()
source_prj.ImportFromProj4(ncfile.GetProjectionRef())
target_prj = osr.SpatialReference()
target_prj.ImportFromProj4("+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs")
# Reproject the data
GeoT = ncfile.GetGeoTransform()
driver = gdal.GetDriverByName('MEM')
raw = driver.Create('raw', array.shape[0], array.shape[1], 1, gdal.GDT_Float32)
raw.SetGeoTransform(GeoT)
raw.GetRasterBand(1).WriteArray(array)
# Define the parameters of the output file
kwargs = {'format': 'netCDF', \
'srcSRS': source_prj, \
'dstSRS': target_prj, \
'outputBounds': (extent[0], extent[3], extent[2], extent[1]), \
'outputBoundsSRS': target_prj, \
'outputType': gdal.GDT_Float32, \
'srcNodata': undef, \
'dstNodata': 'nan', \
'resampleAlg': gdal.GRA_NearestNeighbour}
# Write the reprojected file on disk
gdal.Warp(file_name, raw, **kwargs)